Adaptive Non-Playable Characters with Reinforcement Learning: Training and Shipping a Learned Game Opponent

Authors

  • Imran Maqsood Department of Computer Software Engineering, University of Engineering and Technology Mardan (23200), Pakistan
  • Namos Khan Department of Computer Software Engineering, University of Engineering and Technology Mardan (23200), Pakistan
  • Hijab Noor ul Amin Department of Computer Software Engineering, University of Engineering and Technology Mardan (23200), Pakistan
  • Mareena karim Department of Telecommunication Engineering, University of Engineering and Technology Mardan (23200)
  • Sadeeq Jan National Center for Cyber Security, University of Engineering & Technology, Peshawar, 25000, Pakistan

DOI:

https://doi.org/10.33411/IJIST/1974

Keywords:

Adaptive Non-Playable Characters, Reinforcement Learning, Proximal Policy Optimization, Unity ML-Agents, Artificial Intelligence in Games

Abstract

Most Non-Playable Characters (NPCs) in modern video games still rely on scripted logic, finite state machines, and decision trees. This reliance on hand-authored rules means that their behavior is easy to predict and repeat, which reduces player immersion and long-term engagement. Although reinforcement learning has been proposed as an alternative production method, prior demonstrations are often limited to experimental training environments and seldom implemented in a complete playable prototype. The present study fills this gap by taking a reinforcement-learning-trained NPC through the entire production pipeline, from initial design to a tested, standalone game capable of running on ordinary consumer hardware. We implemented a 3D foraging environment called the Flower Island in Unity, where we trained a hummingbird agent (with a 10-value observation vector, 5 continuous actions and a primary reward based on successful nectar collection) using Proximal Policy Optimization with the Unity ML-Agents toolkit and domain randomization applied throughout training. Training was run until the episodic cumulative reward, monitored in TensorBoard, showed progression from exploratory behaviour to more stable behavioural patterns, followed by a reward plateau, at which point the policy was exported for deployment. Once converged, the policy was exported in ONNX format and executed inside the finished game via Unity's Inference Engine on the Burst CPU backend; in playtesting the exported policy ran in real time without observed frame-rate degradation during playtesting and without inference-related runtime exceptions. This was part of a full title with two different game modes, a full user interface and layered audio system. The resultant policy showed purposeful and diverse foraging behavior and was able to play against a human player in real time. The game passed all ten functional test cases, and the final acceptance test suite ran without any runtime exceptions. Among the engineering factors considered, integration effort, choice of inference backend, and simplicity of the reward structure exerted the greatest influence on the outcome. Formal quantitative benchmarking against a scripted baseline (reward curves, latency figures, and significance testing) was not captured for this iteration of the project and is identified as the primary direction for follow-up work. These results show that adaptive, learned NPCs are possible on commodity hardware and the pipeline described here provides a reproducible template for future games, simulations and educational tools.

References

“Artificial Intelligence for Games - 2nd Edition | Elsevier Shop.” Accessed: Jul. 26, 2026. [Online]. Available: https://shop.elsevier.com/books/artificial-intelligence-for-games/millington/978-0-12-374731-0

G. N. Yannakakis and J. Togelius, “Artificial intelligence and games,” Artif. Intell. Games, pp. 1–337, Feb. 2018, doi: 10.1007/978-3-319-63519-4/COVER.

K. Souchleris, G. K. Sidiropoulos, and G. A. Papakostas, “Reinforcement Learning in Game Industry—Review, Prospects and Challenges,” Appl. Sci. 2023, Vol. 13, Page 2443, vol. 13, no. 4, p. 2443, Feb. 2023, doi: 10.3390/APP13042443.

Sutton, R. S., & Barto, A. G, “Reinforcement learning: An introduction, 2nd ed.,” MIT Press, 2018, [Online]. Available: https://psycnet.apa.org/record/2019-19679-000

T. Wang et al., “player2vec: A Language Modeling Approach to Understand Player Behavior in Games,” Jun. 2024, Accessed: Aug. 04, 2026. [Online]. Available: https://arxiv.org/pdf/2404.04234v3

V. Raj, “A Review on AI-Driven Game Development: Reinforcement Learning and Adaptive NPCs in Modern Games,” Int. J. Res. Appl. Sci. Eng. Technol., vol. 13, no. 12, pp. 3151–3155, Dec. 2025, doi: 10.22214/IJRASET.2025.76668.

M. Korkiakoski, S. Sheikhi, J. Nyman, J. Saariniemi, K. Tapio, and P. Kostakos, “An Empirical Evaluation of AI-Powered Non-Player Characters’ Perceived Realism and Performance in Virtual Reality Environments,” Jul. 2025, Accessed: Aug. 04, 2026. [Online]. Available: https://arxiv.org/pdf/2507.10469

V. R. K, M. Maanas, A. T. S, A. K. P, and R. P. D, “A Review on AI-Driven Game Development: Reinforcement Learning and Adaptive NPCs in Modern Game Design,” Dec. 2025, doi: 10.36227/TECHRXIV.176617583.38277587/V1.

C. Tan, “Comparative Study of Reinforcement Learning Performance Based on PPO and DQN Algorithms,” Appl. Comput. Eng., vol. 175, no. 1, pp. 30–36, Jul. 2025, doi: 10.54254/2755-2721/2025.AST24879.

K. Chrysafiadi, M. Kamitsios, and M. Virvou, “Fuzzy-based dynamic difficulty adjustment of an educational 3D-game,” Multimed. Tools Appl. 2023 8218, vol. 82, no. 18, pp. 27525–27549, Feb. 2023, doi: 10.1007/S11042-023-14515-W.

J. Zhang, “Implementation and effect evaluation of dynamic difficulty adjustment based on reinforcement learning in Multiplayer Online Battle Arena games,” Sci. Technol. Eng. Chem. Environ. Prot., vol. 1, no. 10, Dec. 2024, doi: 10.61173/R8885T34.

P. Zhang, X. Ma, Z. Pan, X. Li, and K. Xie, “Multi-agent cooperative reinforcement learning in 3D virtual world,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 6145 LNCS, no. PART 1, pp. 731–739, 2010, doi: 10.1007/978-3-642-13495-1_90/SAVE-RESEARCH.

R. Duncan, “Using Reinforcement Learning to Train In-game Non-Player Characters (NPCs)”.

A. Juliani et al., “Unity: A General Platform for Intelligent Agents,” Sep. 2018, Accessed: Jul. 26, 2026. [Online]. Available: https://arxiv.org/pdf/1809.02627

J. Orkin, “Three States and a Plan: The A.I. of F.E.A.R.,” 2006.

V. Mnih et al., “Human-level control through deep reinforcement learning,” Nat. 2015 5187540, vol. 518, no. 7540, pp. 529–533, Feb. 2015, doi: 10.1038/nature14236.

D. Silver et al., “Mastering the game of Go with deep neural networks and tree search,” Nat. 2016 5297587, vol. 529, no. 7587, pp. 484–489, Jan. 2016, doi: 10.1038/nature16961.

OpenAI et al., “Dota 2 with Large Scale Deep Reinforcement Learning,” Dec. 2019, Accessed: Jul. 26, 2026. [Online]. Available: https://arxiv.org/pdf/1912.06680

O. Vinyals et al., “Grandmaster level in StarCraft II using multi-agent reinforcement learning,” Nat. 2019 5757782, vol. 575, no. 7782, pp. 350–354, Oct. 2019, doi: 10.1038/s41586-019-1724-z.

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. K. Openai, “Proximal Policy Optimization Algorithms,” Jul. 2017, Accessed: Jul. 26, 2026. [Online]. Available: https://arxiv.org/pdf/1707.06347

S. Fujimoto, H. Van Hoof, and D. Meger, “Addressing Function Approximation Error in Actor-Critic Methods,” 35th Int. Conf. Mach. Learn. ICML 2018, vol. 4, pp. 2587–2601, Feb. 2018, Accessed: Aug. 04, 2026. [Online]. Available: https://arxiv.org/pdf/1802.09477

Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, Sergey Levine, “Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor,” arXiv:1801.01290, 2018, [Online]. Available: https://arxiv.org/abs/1801.01290

V. Mnih et al., “Asynchronous Methods for Deep Reinforcement Learning,” 33rd Int. Conf. Mach. Learn. ICML 2016, vol. 4, pp. 2850–2869, Feb. 2016, Accessed: Aug. 04, 2026. [Online]. Available: https://arxiv.org/pdf/1602.01783

P. Henderson, R. Islam, P. Bachman, J. Pineau, D. Precup, and D. Meger, “Deep Reinforcement Learning that Matters,” 32nd AAAI Conf. Artif. Intell. AAAI 2018, pp. 3207–3214, Sep. 2017, doi: 10.1609/aaai.v32i1.11694.

R. Agarwal, M. Schwarzer, P. S. Castro, A. Courville, and M. G. Bellemare, “Deep Reinforcement Learning at the Edge of the Statistical Precipice,” Adv. Neural Inf. Process. Syst., vol. 35, pp. 29304–29320, Aug. 2021, Accessed: Aug. 04, 2026. [Online]. Available: https://arxiv.org/pdf/2108.13264

“Toolkit Documentation - Unity ML-Agents Toolkit.” Accessed: Jul. 26, 2026. [Online]. Available: https://unity-technologies.github.io/ml-agents/ML-Agents-Toolkit-Documentation/

A. Merritt, “ONNX-Scala: typeful, functional deep learning / Dotty meets an open AI standard (open-source talk),” pp. 33–33, Nov. 2020, doi: 10.1145/3426426.3434120.

“Sentis overview | Sentis | 2.6.1.” Accessed: Jul. 26, 2026. [Online]. Available: https://docs.unity3d.com/Packages/com.unity.ai.inference@2.6/manual/index.html

“Blender 4.0 Reference Manual — Blender Manual.” Accessed: Jul. 26, 2026. [Online]. Available: https://docs.blender.org/manual/en/4.0/

T. Aris, V. Ustun, and R. Kumar, “Learning to Take Cover with Navigation-Based Waypoints via Reinforcement Learning,” Proc. Int. Florida Artif. Intell. Res. Soc. Conf. FLAIRS, vol. 36, 2023, doi: 10.32473/FLAIRS.36.133348.

Downloads

Published

2026-08-14
CITATION
Published: 2026-08-14
Crossref Citation Count: Loading...

How to Cite

Maqsood, I., Khan, N., Noor ul Amin, H., karim, M., & Jan, S. (2026). Adaptive Non-Playable Characters with Reinforcement Learning: Training and Shipping a Learned Game Opponent. International Journal of Innovations in Science & Technology, 8(5), 1903–1916. https://doi.org/10.33411/IJIST/1974