论文标题
在AAA视频游戏中进行导航的深度强化学习
Deep Reinforcement Learning for Navigation in AAA Video Games
论文作者
论文摘要
在视频游戏中，非玩家角色（NPC）用于以多种方式来增强玩家的体验，例如，作为敌人，盟友或无辜的旁观者。 NPC的关键组成部分是导航，这使他们可以从一个点移到地图上的另一点。视频游戏行业中NPC导航的最流行方法是使用导航网格（NavMesh），该导航网格是地图的图表，节点和边缘表明可遍历的区域。不幸的是，扩大角色运动能力的复杂导航能力，例如，抓钩，喷气背包，传送或双重跳跃会增加NavMesh的复杂性，使其在许多实际情况下变得棘手。因此，游戏设计师受到限制，仅添加了NavMesh可以使用NPC导航的能力。作为替代方案，我们建议使用深度加固学习（Deep RL）学习如何使用任何导航能力来浏览3D地图。我们在Unity游戏引擎中的复杂3D环境上测试了我们的方法，该方法显然是比深RL文献中通常使用的地图大的数量级。这些地图之一是在Ubisoft AAA游戏之后直接建模的。我们发现我们的方法表现出色，在所有经过测试的情况下达到至少$ 90 \％$的成功率。我们的结果视频可在https://youtu.be/wfif9wwlq8m上获得。
In video games, non-player characters (NPCs) are used to enhance the players' experience in a variety of ways, e.g., as enemies, allies, or innocent bystanders. A crucial component of NPCs is navigation, which allows them to move from one point to another on the map. The most popular approach for NPC navigation in the video game industry is to use a navigation mesh (NavMesh), which is a graph representation of the map, with nodes and edges indicating traversable areas. Unfortunately, complex navigation abilities that extend the character's capacity for movement, e.g., grappling hooks, jetpacks, teleportation, or double-jumps, increases the complexity of the NavMesh, making it intractable in many practical scenarios. Game designers are thus constrained to only add abilities that can be handled by a NavMesh if they want to have NPC navigation. As an alternative, we propose to use Deep Reinforcement Learning (Deep RL) to learn how to navigate 3D maps using any navigation ability. We test our approach on complex 3D environments in the Unity game engine that are notably an order of magnitude larger than maps typically used in the Deep RL literature. One of these maps is directly modeled after a Ubisoft AAA game. We find that our approach performs surprisingly well, achieving at least $90\%$ success rate on all tested scenarios. A video of our results is available at https://youtu.be/WFIf9Wwlq8M.
