Historically viewed as a niche economic sector, gaming is now projected to exceed a global annual revenue of $ 218.7 billion in 2024, taking advantage of recent Artificial Intelligence (AI) advances. In recent years, specific AI techniques namely; Machine Learning (ML) and Reinforcement Learning (RL), have seen impressive progress and popularity. Techniques developed within these two fields are now able to analyze and learn from gameplay experiences enabling more interactive, immersive, and engaging games. While the number of ML and RL algorithms is growing, their implementations through frameworks and toolkits are also extensive too. Moreover, the game design and development community lacks a framework for informed evaluation of available RL toolkits. In this paper, we present a comprehensive survey of RL toolkits for games using a qualitative evaluation methodology.
Tazouti, Y., Boulaknadel, S. and Fakhri, Y. (2022) Design and Implementation of ImALeG Serious Game: Behavior of Non-Playable Characters (NPC). In: Saeed, F., et al., Eds., Advances on Smart and Soft Computing, Springer, Berlin, 69-77. https://doi.org/10.1007/978-981-16-5559-3_7
Yannakakis, G.N. (2012) Game AI Revisited. Proceedings of the 9th Conference on Computing Frontiers, Caligari, 15-17 May 2012, 285-292. https://doi.org/10.1145/2212908.2212954
Yohanes, D.N. and Rochmawati, N. (2022) Implementasi Algoritma Collision Detection dan A*(A Star) pada Non Player Character Game World of New Normal. Journal of Informatics and Computer Science, 3, 322-333. https://doi.org/10.26740/jinacs.v3n03.p322-333
Frank, A.B. (2022) Gaming AI without AI. The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology. https://doi.org/10.1177/15485129221074352
Lyle, D., et al. (2022) Chess and Strategy in the Age of Artificial Intelligence. In: Lai, D., Ed., US-China Strategic Relations and Competitive Sports, Springer, Berlin, 87-126. https://doi.org/10.1007/978-3-030-92200-9_5
Sweetser, P. and Wiles, J. (2002) Current AI in Games: A Review. Australian Journal of Intelligent Information Processing Systems, 8, 24-42.
Yannakakis, G.N. and Togelius, J. (2014) A Panorama of Artificial and Computational Intelligence in Games. IEEE Transactions on Computational Intelligence and AI in Games, 7, 317-335. https://doi.org/10.1109/TCIAIG.2014.2339221
Shao, K., Tang, Z., Zhu, Y., Li, N. and Zhao, D. (2019) A Survey of Deep Reinforcement Learning in Video Games.
Palma-Ruiz, J.M., Torres-Toukoumidis, A., Gonzalez-Moreno, S.E. and Valles-Baca, H.G. (2022) An Overview of the Gaming Industry across Nations: Using Analytics with Power Bi to Forecast and Identify Key Influencers. Heliyon, 8, e08959. https://doi.org/10.1016/j.heliyon.2022.e08959
Bornemark, O. (2013) Success Factors for e-Sport Games. Umeå’s 16th Student Conference in Computing Science, 1-12.
Gonzalez-Moreno, M.S.E., Montalvo, J.A.C. and Palma-Ruiz, J.M. (2019) La industria cultural y la industria de los videojuegos. In: Juegos y Sociedad: Desde La Interaccio’N a la Inmersion Para el Cambio Social, McGraw Hill, New York, 19-26.
Li, R. (2017) Good Luck Have Fun: The Rise of eSports. Simon and Schuster, New York.
Borowy, M., et al. (2013) Pioneering eSports: The Experience Economy and the Marketing of Early 1980s Arcade Gaming Contests. International Journal of Communication, 7, 2254-2275.
Saiz-Alvarez, J.M., Palma-Ruiz, J.M., Valles-Baca, H.G. and Fierro-Ramırez, L.A. (2021) Knowledge Management in the eSports Industry: Sustainability, Continuity, and Achievement of Competitive Results. Sustainability, 13, Article No. 10890. https://doi.org/10.3390/su131910890
Scholz, T.M., Scholz, T.M. and Barlow (2019) eSports Is Business. Springer, Berlin. https://doi.org/10.1007/978-3-030-11199-1
Jorda, M.I. and Mitchell, T.M. (2015) Machine Learning: Trends, Perspectives, and Prospects. Science, 349, 255-260. https://doi.org/10.1126/science.aaa8415
Bertens, P., Guitart, A., Chen, P.P. and Perianez, A. (2018) A Machine-Learning Item Recommendation System for Video Games. 2018 IEEE Conference on Computational Intelligence and Games, Maastricht, 14-17 August 2018, 1-4. https://doi.org/10.1109/CIG.2018.8490456
Vondrek, M., Baggili, I., Casey, P. and Mekni, M. (2022) Rise of the Metaverse’s Immersive Virtual Reality Malware and the Man-in-the-Room Attack & Defenses. Computers & Security, 238, Article ID: 102923. https://doi.org/10.1016/j.cose.2022.102923
Tucker, A., Gleave, A. and Russell, S. (2018) Inverse Reinforcement Learning for Video Games.
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J. and Zaremba, W. (2016) Openai Gym.
Duryea, E., Ganger, M. and Hu, W. (2016) Exploring Deep Reinforcement Learning with Multi q-Learning. Intelligent Control and Automation, 7, 129-144. https://doi.org/10.4236/ica.2016.74012
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D. and Riedmiller, M. (2013) Playing Atari with Deep Reinforcement Learning.
Silver, D., Hubert, T., Schrittwieser, J., et al. (2018) A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go through Self-Play. Science, 362, 1140-1144. https://doi.org/10.1126/science.aar6404
Samara, F., Ondieki, S., Hossain, A.M. and Mekni, M. (2021) Online Social Network Interactions (OSNI): A Novel Online Reputation Management Solution. 2021 IEEE International Conference on Engineering and Emerging Technologies, Istanbul, 27-28 October 2021, 1-6. https://doi.org/10.1109/ICEET53442.2021.9659615
Schrittwieser, J., Antonoglou, I., Hubert, T., et al. (2020) Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. Nature, 588, 604-609. https://doi.org/10.1038/s41586-020-03051-4
Andrew, A.M. (1999) Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto, Adaptive Computation and Machine Learning Series, MIT Press (Bradford Book), Cambridge, Mass., 1998, xviii+ 322 pp, ISBN 0-262-19398-1, (Hardback, ￡ 31.95). Robotica, 17, 229-235. https://doi.org/10.1017/S0263574799211174
Silver, D., Huang, A., Maddison, C.J., et al. (2016) Mastering the Game of Go with Deep Neural Networks and Tree Search. Nature, 529, 484-489. https://doi.org/10.1038/nature16961
Silver, D., Schrittwieser, J., Simonyan, K., et al. (2017) Mastering the Game of Go without Human Knowledge. Nature, 550, 354-359. https://doi.org/10.1038/nature24270
Arulkumaran, K., Cully, A. and Togelius, J. (2019) Alphastar: An Evolutionary Computation Perspective. Proceedings of the Genetic and Evolutionary Computation Conference Companion, Prague, 13-17 July 2019, 314-315. https://doi.org/10.1145/3319619.3321894
Berner, C., Brockman, G., Chan, B., et al. (2019) Dota 2 with Large Scale Deep Reinforcement Learning.
Sweeney, N. and Sinclair, D. (2012) Applying Reinforcement Learning to Poker. Computer Poker Symposium, Quebec, 314-315.
Nandy, A. and Biswas, M. (2018) Machine Learning Agents and Neural Network in Unity. In: Nandy, A. and Biswas, M., Eds., Neural Networks in Unity, Springer, Berlin, 69-111. https://doi.org/10.1007/978-1-4842-3673-4_3
Jayaramireddy, C.S., Naraharisetti, S.V., Nassar, M. and Mekni, M. (2023) A Survey of Reinforcement Learning Toolkits for Gaming: Applications, Challenges and Trends. In: Arai, K., Ed., Proceedings of the Future Technologies Conference, Springer, Berlin, 165-184. https://doi.org/10.1007/978-3-031-18461-1_11
Lanham, M. (2018) Learn Unity ML-Agents-Fundamentals of Unity Machine Learning: Incorporate New Powerful ML Algorithms Such as Deep Reinforcement Learning for Games. Packt Publishing Ltd., Birmingham.
Juliani, A., Berges, V.-P., Teng, E., et al. (2018) Unity: A General Platform for Intelligent Agents.
Baby, N. and Goswami, B. (2019) Implementing Artificial Intelligence Agent within Connect 4 Using Unity3D and Machine Learning Concepts. International Journal of Recent Technology and Engineering, 7, 193-200.
Cao, Z. and Lin, C.-T. (2021) Reinforcement Learning from Hierarchical Critics. IEEE Transactions on Neural Networks and Learning Systems, 1-8. https://doi.org/10.1109/TNNLS.2021.3103642
Borovikov, I., Harder, J., Sadovsky, M. and Beirami, A. (2019) Towards Inter-Active Training of Non-Player Characters in Video Games.
Silver, T. and Chitnis, R. (2020) Pddlgym: Gym Environments from Pddl Problems.
Ray, A., Achiam, J. and Amodei, D. (2019) Benchmarking Safe Exploration in Deep Reinforcement Learning. Vol. 7.
Nichol, A., Pfau, V., Hesse, C., Klimov, O. and Schulman, J. (2018) Gotta Learn Fast: A New Benchmark for Generalization in rl.
Terry, J., Black, B., Grammel, N., et al. (2021) PettingZoo: Gym for Multi-Agent Reinforcement Learning. Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021), London, 3-7 May 2021, 441-470.
Castro, P.S., Moitra, S., Gelada, C., et al. (2018) Dopamine: A Research Framework for Deep Reinforcement Learning. http://arxiv.org/abs/1812.06110
Booth, J. and Booth, J. (2019) Marathon Environments: Multi-Agent Continuous Control Benchmarks in a Modern Video Game Engine.
Nowe, A., Vrancx, P. and Hauwere, Y.-M.D. (2012) Game Theory and Multiagent Reinforcement Learning. In: Wiering, M. and Otterlo, M., Eds., Reinforcement Learning, Springer, Berlin, 441-470. https://doi.org/10.1007/978-3-642-27645-3_14
Lowe, R., Wu, Y., Tamar, A., et al. (2017) Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments.