Friday November 12 11:35 – Friday November 12 12:10 in Planck Bohr

MimicBot: Combining Imitation and Reinforcement Learning to win the Bot Bowl competition

Nicola Pezzotti

Prior knowledge:
No previous knowledge expected

Summary

In this talk, I will introduce you to MimicBot, a reinforcement learning agent that uses imitation learning to play Blood Bowl. MimicBot won the Bot Bowl III competition and it is the first machine learning model to beat a scripted AI agent.

Outline

Description

This paper describe an hybrid agent trained to play in Fantasy Football AI which participated in the Bot Bowl III competition. The agent, MimicBot, is implemented using a specifically designed deep policy network and trained using a combination of imitation and reinforcement learning. Previous attempts in using a reinforcement learning approach in such context failed for a number of reasons, e.g. due to the intrinsic randomness in the environment and the large and uneven number of actions available, with a curriculum learning approach failing to consistently beat a randomly paying agent. Currently no machine learning approach can beat a scripted bot which makes use of the domain knowledge on the game. Our solution, thanks to an imitation learning and a hybrid decision-making process, consistently beat such scripted agents. Moreover we shed lights on how to more efficiently train in a reinforcement learning setting while drastically increasing sample efficiency. MimicBot is the winner of the Bot Bowl III competition, and it is currently the state-of-the-art solution.