CCRI: Planning-C: Developing a Minecraft-based Testbed for Evaluating Human-AI Teaming Research

人类人工智能团队(HAT)是一个新兴且快速发展的领域，其中物理和虚拟空间需要快速的战略和战术决策。在国家安全方面，必须开发能够加快所谓观察到行动循环的工具，使决策者能够迅速和有效地采取行动，减少我们的敌对威胁的影响。团队在动态和不确定的环境中运作是很常见的，他们不仅需要人类和人工代理之间的合作，而且需要使用人工智能(AI)来帮助构建这些团队，创建行动计划，执行这些计划，并随着环境和目标的变化而调整计划。人工和人工代理之间的这种集成可用于提高人类决策者的速度和准确性；自动化任务计划的生成、评估和执行；并提高培训有效性。这笔CEISE计算研究基础设施(CCRI)规划赠款将进行以社区为中心的详尽需求收集、测试和最终确定过程，这将产生一套全面的功能，以构建由工具组成的软件套件，以提高人工智能团队的效率。具体地说，拟议的软件试验床将是灵活和可定制的，以满足广泛的人工智能研究人员的需求，并帮助他们加快开发设计用于人类人工智能团队的人工智能代理。通过将人工智能社区组织成多个工作组并通过研讨会传播结果，该项目旨在制定修改广受欢迎的我的世界游戏平台的要求，包括将在HAT研究人员社区内实现预期目标的工具的数据工程、测试和评估要求。该团队在之前成功使用我的世界在基于城市搜索和救援(USR)的环境中进行HAT实验的基础上，扩大了它在对开发和测试HAT实验感兴趣的人工智能研究人员中的使用。由此产生的社区反馈将被整合到CCRI的全面提案中，该提案旨在根据通过规划拨款进行的活动所产生的要求来构建软件套件。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Human AI Teaming (HAT) is an emerging and rapidly evolving field where physical and virtual spaces require rapid strategic and tactical decision-making. In the context of national security, it is imperative to develop tools that can accelerate what is known as the observation to action loop, enabling decision makers to take swift and effective action that can reduce the impacts of our adversarial threats. It is common for teams to operate in dynamic and uncertain environments and they will require not only cooperation between human and artificial agents, but the use of artificial intelligence (AI) to help structure those teams, create plans of actions, execute those plans, and adapt plans as the environment and goals change. This integration between human and artificial agents can be used to enhance the speed and accuracy of human decision makers; automate the generation, evaluation, and execution of mission plans; and improve training effectiveness. This CISE Computing Research Infrastructure (CCRI) planning grant will undertake an exhaustive community-centric requirements gathering, testing and finalization process that would result in a comprehensive set of features necessary to build a software suite comprising of tools that would improve the effectiveness of human AI teams. Specifically, the proposed software testbed would be flexible and customizable to meet the needs of a broad community of AI researchers and helps them accelerate the development of AI agents that are designed to work in Human AI teams. By organizing the AI community into multiple working groups and disseminating the results via a workshop, the project aims to develop the requirements for modifying the popular Minecraft gaming platform including the data engineering, testing and evaluation requirements for tools that would achieve the desired goals within the HAT community of researchers. The team builds upon prior success in using Minecraft for conducting HAT experiments in Urban Search and Rescue (USR) based settings and broaden its use among AI researchers who would be interested in developing and testing HAT experiments. The resulting community feedback would be integrated into a CCRI full proposal that aims to build the software suite based on the requirements arising from the activities conducted via the planning grant.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.