NSF2026: EAGER:Cues and actions for efficient nonverbal human-robot communication

很大一部分人类群体交流是通过非语言进行的。人们跟着凝视，避免碰撞，以团队形式进行搜索和救援，所有这些都是不说话的。在这方面，行动而不是语言能够实现快速的双向信息交流，而不会干扰手头的任务，也不会造成理解语音和文本所需的额外心理负担。整合人机智能将受益于人与机器之间类似的自然和流畅的交流。该项目通过一系列实验研究和严格的数学分析，开发了促进人类-机器人智能的新方法。这些实验涉及旨在利用机器人和人类优势的任务；机器人能够重复探索大环境，人类对情况和领域专业知识有更好的认识。实验任务的灵感来自于监测威胁大湖区的大量入侵水生物种的难题。数学分析旨在发现机器人响应人类认知负荷变化的有效动作，以及人与机器人之间有效的非语言交互策略。这项工作的研究成果将提高公众对大湖区入侵水生物种的认识，并将人类-机器人合作作为解决大规模问题的重要机会。参与该项目的工科学生将为新一代科学劳动力做出贡献，他们可以跨越机器人学、计算机科学和生态学等多个学科的边界。这项研究旨在通过梳理出有效的非语言人-机器人交流的组成部分，使人-机器人智能更紧密地整合在一起。这些包括：(I)机器人群的参与程度作为人类认知负荷的函数；(Ii)人类在与机器人群合作绘制复杂动态环境地图时所使用的招募策略；(Iii)人类对机器人集群模式的感知；以及(Iv)能够在野外更快地解释人类认知负荷的间接指标。对此，实验条件将突出团队表现对机器人如何应对人类参与者所经历的认知负荷的依赖。实验将在虚拟现实中进行，以实现大规模的机器人群，而不会伴随着设计和传感器编程的挑战。通过建立基于本地交互规则的所有环境感知和交互策略，将保持群机器人的可扩展性。定向信息流的信息论测量将被用来量化人类对蜂群模式的感知，并分离认知负荷的运动关联。该项目得到了CEISE局IIS司的以人为中心的计算计划和综合活动办公室的NSF 2026基金计划的支持。该项目丰富、扩展和探索了NSF 2026 Idea Machine获奖作品“集成人机智能”。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

A large part of human group communication takes place nonverbally. People follow gaze, avoid collisions, search and rescue in teams, all without speaking to each other. In this respect, actions, rather than words, enable rapid two-way communication of information without interfering with the task at hand or posing additional mental burden required to understand speech and text. Integrating human-machine intelligence would benefit from a similar natural and fluid communication between humans and machines. This project develops novel methods to advance human-robot intelligence through a series of experimental studies and rigorous mathematical analysis. The experiments involve tasks designed to exploit the strengths of robots and humans; robots are able to repetitively explore a large environment and humans have better awareness of the situation and domain expertise. The experimental tasks are inspired by the difficult problem of monitoring the vast number of invasive aquatic species threatening the Great Lakes region. The mathematical analysis is aimed at discovering effective robot actions in response to changes in human cognitive load, and efficient nonverbal interaction strategies between humans and robots. Research results from this work will raise public awareness of invasive aquatic species in the Great Lakes region and present human-robot teaming as a prominent opportunity to solve large-scale problems. Engineering students involved in the project will contribute to the new generation of scientific workforce who can straddle boundaries across multiple disciplines such as robotics, computer science, and ecology.This research aims to enable tighter integration of human-robot intelligence by teasing out components of efficient nonverbal human-robot communication. These include: (i) level of engagement of the robotic swarm as a function of human cognitive load, (ii) recruitment strategies used by humans as they team up with the robotic swarm to map a complex dynamic environment, (iii) perception of robot swarming patterns by humans, and (iv) indirect indicators of human cognitive load that can enable faster interpretation in the wild. Towards this, experimental conditions will highlight the dependence of team performance on how robots respond to the cognitive load experienced by the human participants. Experiments will be conducted in virtual reality to enable realization of large robot swarms without the accompanying design and sensor programming challenges. Scalability of swarm robotics will be preserved by building all environmental sensing and interaction strategies upon local interaction rules. Information-theoretic measures of directional information flow will be used to quantify human perception of swarm patterns and isolate movement correlates of cognitive load. This project has the support of the Human-Centered-Computing Program in the IIS Division in the CISE Directorate, and the NSF 2026 Fund Program in the Office of Integrated Activities. The project enriches, extends, and explores the NSF 2026 Idea Machine Winning Entry “Integrated Human-Machine Intelligence”.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.