Author: Rob Matheson | MIT News Office<\/p>\n
MIT researchers have developed a bot equipped with artificial intelligence that can beat human players in tricky online multiplayer games where player roles and motives are kept secret.<\/p>\n
Many gaming bots have been built to keep up with human players. Earlier this year, a team from Carnegie Mellon University developed the world\u2019s first bot that can beat professionals in multiplayer poker. DeepMind\u2019s AlphaGo made headlines in 2016 for besting a professional Go player. Several bots have also been built to beat professional chess players or join forces in cooperative games such as online capture the flag. In these games, however, the bot knows its opponents and teammates from the start.<\/p>\n
At the Conference on Neural Information Processing Systems next month, the researchers will present DeepRole, the first gaming bot that can win online multiplayer games in which the participants\u2019 team allegiances are initially unclear. The bot is designed with novel \u201cdeductive reasoning\u201d added into an AI algorithm commonly used for playing poker. This helps it reason about partially observable actions, to determine the probability that a given player is a teammate or opponent. In doing so, it quickly learns whom to ally with and which actions to take to ensure its team\u2019s victory.<\/p>\n
The researchers pitted DeepRole against human players in more than 4,000 rounds of the online game \u201cThe Resistance: Avalon.\u201d In this game, players try to deduce their peers\u2019 secret roles as the game progresses, while simultaneously hiding their own roles. As both a teammate and an opponent, DeepRole consistently outperformed human players.<\/p>\n
\u201cIf you replace a human teammate with a bot, you can expect a higher win rate for your team. Bots are better partners,\u201d says first author Jack Serrino \u201918, who majored in electrical engineering and computer science at MIT and is an avid online \u201cAvalon\u201d player.<\/p>\n
The work is part of a broader project to better model how humans make socially informed decisions. Doing so could help build robots that better understand, learn from, and work with humans.<\/p>\n
\u201cHumans learn from and cooperate with others, and that enables us to achieve together things that none of us can achieve alone,\u201d says co-author Max Kleiman-Weiner, a postdoc in the Center for Brains, Minds and Machines and the Department of Brain and Cognitive Sciences at MIT, and at Harvard University. \u201cGames like \u2018Avalon\u2019 better mimic the dynamic social settings humans experience in everyday life. You have to figure out who\u2019s on your team and will work with you, whether it\u2019s your first day of kindergarten or another day in your office.\u201d<\/p>\n
Joining Serrino and Kleiman-Weiner on the paper are David C. Parkes of Harvard and Joshua B. Tenenbaum, a professor of computational cognitive science and a member of MIT\u2019s Computer Science and Artificial Intelligence Laboratory and the Center for Brains, Minds and Machines.<\/p>\n
Deductive bot<\/strong><\/p>\n In \u201cAvalon,\u201d three players are randomly and secretly assigned to a \u201cresistance\u201d team and two players to a \u201cspy\u201d team. Both spy players know all players\u2019 roles. During each round, one player proposes a subset of two or three players to execute a mission. All players simultaneously and publicly vote to approve or disapprove the subset. If a majority approve, the subset secretly determines whether the mission will succeed or fail. If two \u201csucceeds\u201d are chosen, the mission succeeds; if one \u201cfail\u201d is selected, the mission fails. Resistance players must always choose to succeed, but spy players may choose either outcome. The resistance team wins after three successful missions; the spy team wins after three failed missions.<\/p>\n Winning the game basically comes down to deducing who is resistance or spy, and voting for your collaborators. But that\u2019s actually more computationally complex than playing chess and poker. \u201cIt\u2019s a game of imperfect information,\u201d Kleiman-Weiner says. \u201cYou\u2019re not even sure who you\u2019re against when you start, so there\u2019s an additional discovery phase of finding whom to cooperate with.\u201d<\/p>\n DeepRole uses a game-planning algorithm called \u201ccounterfactual regret minimization\u201d (CFR) \u2014 which learns to play a game by repeatedly playing against itself \u2014 augmented with deductive reasoning. At each point in a game, CFR looks ahead to create a decision \u201cgame tree\u201d of lines and nodes describing the potential future actions of each player. Game trees represent all possible actions (lines) each player can take at each future decision point. In playing out potentially billions of game simulations, CFR notes which actions had increased or decreased its chances of winning, and iteratively revises its strategy to include more good decisions. Eventually, it plans an optimal strategy that, at worst, ties against any opponent.<\/p>\n CFR works well for games like poker, with public actions \u2014 such as betting money and folding a hand \u2014 but it struggles when actions are secret. The researchers\u2019 CFR combines public actions and consequences of private actions to determine if players are resistance or spy.<\/p>\n The bot is trained by playing against itself as both resistance and spy. When playing an online game, it uses its game tree to estimate what each player is going to do. The game tree represents a strategy that gives each player the highest likelihood to win as an assigned role. The tree\u2019s nodes contain \u201ccounterfactual values,\u201d which are basically estimates for a payoff that player receives if they play that given strategy.<\/p>\n At each mission, the bot looks at how each person played in comparison to the game tree. If, throughout the game, a player makes enough decisions that are inconsistent with the bot\u2019s expectations, then the player is probably playing as the other role. Eventually, the bot assigns a high probability for each player\u2019s role. These probabilities are used to update the bot\u2019s strategy to increase its chances of victory.<\/p>\n Simultaneously, it uses this same technique to estimate how a third-person observer might interpret its own actions. This helps it estimate how other players may react, helping it make more intelligent decisions. \u201cIf it\u2019s on a two-player mission that fails, the other players know one player is a spy. The bot probably won\u2019t propose the same team on future missions, since it knows the other players think it\u2019s bad,\u201d Serrino says.<\/p>\n Language: The next frontier<\/strong><\/p>\n Interestingly, the bot did not need to communicate with other players, which is usually a key component of the game. \u201cAvalon\u201d enables players to chat on a text module during the game. \u201cBut it turns out our bot was able to work well with a team of other humans while only observing player actions,\u201d Kleiman-Weiner says. \u201cThis is interesting, because one might think games like this require complicated communication strategies.\u201d<\/p>\n Next, the researchers may enable the bot to communicate during games with simple text, such as saying a player is good or bad. That would involve assigning text to the correlated probability that a player is resistance or spy, which the bot already uses to make its decisions. Beyond that, a future bot might be equipped with more complex communication capabilities, enabling it to play language-heavy social-deduction games \u2014 such as a popular game \u201cWerewolf\u201d \u2014which involve several minutes of arguing and persuading other players about who\u2019s on the good and bad teams.<\/p>\n \u201cLanguage is definitely the next frontier,\u201d Serrino says. \u201cBut there are many challenges to attack in those games, where communication is so key.\u201d<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"