Aprendizado por reforço assistido por imitação para jogos digitais

Abstract

Reinforcement Learning (RL) and Imitation Learning (IL) are branches of Artificial Intelligence that enable learning through interaction with the environment and through observation of examples, respectively. They have applications in several areas, such as: autonomous vehicles, robot control and games. Games are widely used to test the performance of Reinforcement Learning models, usually using deep neural networks, as they provide a controlled environment capable of exposing the model to a wide variety of problems and contexts. Thus, the present work aims to propose control models for the game Sonic The Hedgehog using Imitation Learning and Deep Reinforcement Learning. In addition, we seek to analyze the performance of imitation models based on adversarial strategies, investigate the impact of imitation on the model’s behavior and performance, and verify whether Imitation Learning can be a viable alternative to creating reward functions. Experiments were carried out comparing different IL methods, in order to verify if it would be able to generate good controllers for the game. Then, the IL methods of behavioral cloning, Adversarial Generative Imitation Learning and Adversarial Inverse Reinforcement Learning were used to start the RL, with the hypothesis that the prior domain knowledge provided by imitation helps the model to achieve better results. The obtained results showed that the IL can be used to generate digital game controllers and that the initialization of the RL step with Imitation Learning can help the model to obtain better performance.