This week is AIIDE, a big academic conference all about AI and games. For the last few years I’ve co-organised a workshop called EXAG along with Alex Zook and Antonios Liapis, and this weekend it’ll be happening again. EXAG is always a very special time of year for me, and the papers I put into EXAG are normally my most favourite out of the whole year, because they can be about all kinds of new and unusual things. This year I wrote one with Adam Summerville about DOTA 2, and I’d like to tell you a little bit about the paper and the game.
The crux of our paper is this: we wanted to be able to predict which hero would be chosen next by professional teams playing in a tournament. At the start of a game of professional-tier DOTA 2 teams pick heroes to be on their team and ban heroes from being in the game altogether. This process of taking turns picking and banning heroes is called drafting, and it’s so vital and full of strategy that teams compile ‘bibles’ of drafting strategy, historical data about opponents, and statistics about hero winrates. Some teams even hire staff to pore over previous games and provide statistical insight.
Compared to DOTA itself – a hectic mess of colours and particle effects and critters – drafting is serene and patient. Teams are given a time limit to make each decision, and a pool of bonus time they can dip into if they need a bit longer to think. Eventually they make a choice, and the draft moves to the next decision point. Drafting isn’t a case of simply picking the ‘best’ heroes (most of the time). It’s about building a strategy you feel confident in executing, about defending yourself against an enemy strategy, about banning out heroes the enemy needs to succeed, and equally banning out heroes that might stop your own strategy succeeding.
For a taste of how a player’s mindset works during drafting here’s Peter Dager, captain of the team who won the 2015 International, talking about a game played at the 2016 International. It’s 12 minutes long and includes a lot of jargon, but you’ll get a flavour:
Drafting is great. I can talk for hours about drafting. Drafting is actually more interesting to me than the game itself sometimes. Predicting what comes next in a draft is something spectators do, commentators do, analysts do, players do, and managers do. Everyone is invested in what might be coming next, because it can change the entire draft, and as a result the entire game. So a tool that can help predict hero picks can help train players to draft better, help commentators get more powerful insights into games, or help provide better in-game spectator tools.
I’m no expert on machine learning, but the very talented Adam Summerville certainly is, and after a long night’s talking about DOTA 2 at Banff in Canada earlier this year, we decided to team up on the project. Adam played DOTA back before it had a 2 in its name, and was incredibly enthusiastic, always looking for more data, developing new system structures and explaining to me the latest tricks he was using. I learned a lot about machine learning from this project, even though I still feel mostly hopeless. In the end, Adam developed a pretty impressive system that could perform to quite good levels. We did some comparisons with human experts by transcribing DOTA draft commentaries and found that we’re not far off human levels of accuracy.
One weakness the system has right now is that when it’s wrong it can be wrong in quite spectacular ways. Often analysts would identify good hero picks, but get the guess wrong because they are unfamiliar with the style of DOTA the team in question plays, for example. When our system hiccups it’s sometimes harder for it to explain why it chose a hero as its top pick, and occasionally there wouldn’t be any obvious reason for it. Knowing machine learning systems, I imagine there is a sophisticated reason for many of its errors, but without being aware of them it can look a bit strange. But we can work on that for another paper!
I’m hugely grateful to Adam for working with me on this and doing such incredible work. If you’d like to check out the paper, I’ve put a PDF online here, and there might be a stream of Adam’s presentation online soon. Watch out for more DOTA 2 stuff from us in the future, hopefully!
One thought on “The First Draft – DOTA 2 & EXAG”