CRII: RI: A Deep Gameplay Framework for Strong Story Experience Management

该奖项全部或部分由《2021年美国救援计划法案》（公法117-2）资助。体验管理者是一种智能代理，能够根据玩家在数字游戏中的决策而创造个性化故事。这些代理为艺术和娱乐、培训应用和个性化教育创造了新的强大的互动故事类型。体验管理者面临的一个核心问题是避免死胡同，即由于玩家的选择而导致故事结构被破坏的情况。这个问题会导致玩家体验到无趣的故事，错过训练序列，以及没有适当目标教育内容的长时间咒语。该项目将开发一种新颖的体验管理架构，以便在与人类参与者的实时交互过程中快速绕过死胡同。该架构基于一般游戏中深度强化学习的最新进展。这个经验管理平台将实现新形式的实时培训和教育应用。这项工作解决了现有体验管理器的一个基本缺陷，即没有针对导致死胡同的玩家行动序列进行对立性计划。具体的研究目标是创建一个基于深度强化学习的游戏代理平台，该平台(1)构建由动作语言描述的状态空间，(2)在不执行穷举搜索的情况下识别死角状态，(3)放松零和对称游戏玩法的假设，以及(4)通过将专门的叙事推理编译成标准的动作语言领域描述来解决叙事规划问题。如果成功，本研究将显著提高体验管理代理的速度和控制，并为控制现有和未来的专业互动叙事形式提供管道。与当前系统相比，这些改进将允许控制更大、更沉浸式的叙事、训练和教学环境。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Experience managers are intelligent agents that produce personalized stories that change based on decisions a player makes in a digital game. These agents create new and powerful types of interactive stories for art and entertainment, training applications, and personalized education. A central problem for experience managers is avoiding dead ends, which are situations where story structure is broken due to a choice by the player. This problem results in experiences with un-interesting stories, missed training sequences, and long spells without appropriate targeted educational content. This project will develop a novel experience management architecture to quickly navigate around dead end situations during real-time interaction with a human participant. The architecture is based on recent advances in deep reinforcement learning for general game playing. This experience management platform will enable new forms of real-time training and education applications.This work addresses a fundamental gap in existing experience managers that do not adversarially plan against sequences of player actions that lead to dead end situations. The specific research objectives are to create a deep reinforcement learning-based gameplay agent platform that (1) builds state spaces described by an action language, (2) identifies dead end states without performing exhaustive search, (3) relaxes assumptions of zero-sum and symmetric gameplay, and (4) solves narrative planning problems by compiling specialized narrative reasoning into a standard action language domain description. If successful, this research will significantly improve the speed and control of experience management agents and provide a pipeline to controlling existing and future specialized interactive narrative formalisms. These improvements will allow control of larger and more immersive narrative, training, and pedagogical environments compared to current systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.