NSF2026: EAGER:Cues and actions for efficient nonverbal human-robot communication

人类群体交流的很大一部分是非语言的。人们跟随目光，避免碰撞，在团队中搜索和救援，所有这些都不需要彼此说话。在这方面，行动，而不是言语，能够实现信息的快速双向交流，而不会干扰手头的任务，也不会给理解言语和文本带来额外的精神负担。整合人机智能将受益于人与机器之间类似的自然和流畅的交流。该项目通过一系列实验研究和严格的数学分析，开发了新的方法来推进人机智能。这些实验涉及的任务旨在利用机器人和人类的优势；机器人能够重复地探索一个大的环境，人类有更好的情况意识和领域的专业知识。实验任务的灵感来自于监测威胁大湖地区的大量入侵水生物种的难题。数学分析的目的是发现机器人在应对人类认知负荷变化时的有效动作，以及人与机器人之间有效的非语言互动策略。这项工作的研究成果将提高公众对大湖地区入侵水生物种的认识，并将人机合作作为解决大规模问题的重要机会。参与该项目的工科学生将为新一代的科学工作者做出贡献，他们可以跨越多个学科的边界，如机器人、计算机科学和生态学。本研究旨在通过梳理有效的非语言人机交流的组成部分，使人机智能更加紧密地集成。这些包括：(i)机器人群体的参与水平作为人类认知负荷的函数，（ii）人类与机器人群体合作绘制复杂动态环境时使用的招聘策略，（iii）人类对机器人群体模式的感知，以及（iv）人类认知负荷的间接指标，可以在野外实现更快的解释。为此，实验条件将突出团队表现对机器人如何应对人类参与者所经历的认知负荷的依赖。实验将在虚拟现实中进行，以实现大型机器人群，而无需伴随的设计和传感器编程挑战。通过建立基于局部交互规则的所有环境感知和交互策略，可以保持群体机器人的可扩展性。定向信息流的信息论措施将用于量化人类对群体模式的感知，并隔离运动相关的认知负荷。本项目得到了CISE理事会IIS部门的以人为中心的计算项目和综合活动办公室的NSF 2026基金项目的支持。该项目丰富、扩展和探索了NSF 2026 Idea Machine获奖项目“集成人机智能”。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

A large part of human group communication takes place nonverbally. People follow gaze, avoid collisions, search and rescue in teams, all without speaking to each other. In this respect, actions, rather than words, enable rapid two-way communication of information without interfering with the task at hand or posing additional mental burden required to understand speech and text. Integrating human-machine intelligence would benefit from a similar natural and fluid communication between humans and machines. This project develops novel methods to advance human-robot intelligence through a series of experimental studies and rigorous mathematical analysis. The experiments involve tasks designed to exploit the strengths of robots and humans; robots are able to repetitively explore a large environment and humans have better awareness of the situation and domain expertise. The experimental tasks are inspired by the difficult problem of monitoring the vast number of invasive aquatic species threatening the Great Lakes region. The mathematical analysis is aimed at discovering effective robot actions in response to changes in human cognitive load, and efficient nonverbal interaction strategies between humans and robots. Research results from this work will raise public awareness of invasive aquatic species in the Great Lakes region and present human-robot teaming as a prominent opportunity to solve large-scale problems. Engineering students involved in the project will contribute to the new generation of scientific workforce who can straddle boundaries across multiple disciplines such as robotics, computer science, and ecology.This research aims to enable tighter integration of human-robot intelligence by teasing out components of efficient nonverbal human-robot communication. These include: (i) level of engagement of the robotic swarm as a function of human cognitive load, (ii) recruitment strategies used by humans as they team up with the robotic swarm to map a complex dynamic environment, (iii) perception of robot swarming patterns by humans, and (iv) indirect indicators of human cognitive load that can enable faster interpretation in the wild. Towards this, experimental conditions will highlight the dependence of team performance on how robots respond to the cognitive load experienced by the human participants. Experiments will be conducted in virtual reality to enable realization of large robot swarms without the accompanying design and sensor programming challenges. Scalability of swarm robotics will be preserved by building all environmental sensing and interaction strategies upon local interaction rules. Information-theoretic measures of directional information flow will be used to quantify human perception of swarm patterns and isolate movement correlates of cognitive load. This project has the support of the Human-Centered-Computing Program in the IIS Division in the CISE Directorate, and the NSF 2026 Fund Program in the Office of Integrated Activities. The project enriches, extends, and explores the NSF 2026 Idea Machine Winning Entry “Integrated Human-Machine Intelligence”.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.