NSF2026: EAGER:Cues and actions for efficient nonverbal human-robot communication

点击翻译按钮获取中文摘要

A large part of human group communication takes place nonverbally. People follow gaze, avoid collisions, search and rescue in teams, all without speaking to each other. In this respect, actions, rather than words, enable rapid two-way communication of information without interfering with the task at hand or posing additional mental burden required to understand speech and text. Integrating human-machine intelligence would benefit from a similar natural and fluid communication between humans and machines. This project develops novel methods to advance human-robot intelligence through a series of experimental studies and rigorous mathematical analysis. The experiments involve tasks designed to exploit the strengths of robots and humans; robots are able to repetitively explore a large environment and humans have better awareness of the situation and domain expertise. The experimental tasks are inspired by the difficult problem of monitoring the vast number of invasive aquatic species threatening the Great Lakes region. The mathematical analysis is aimed at discovering effective robot actions in response to changes in human cognitive load, and efficient nonverbal interaction strategies between humans and robots. Research results from this work will raise public awareness of invasive aquatic species in the Great Lakes region and present human-robot teaming as a prominent opportunity to solve large-scale problems. Engineering students involved in the project will contribute to the new generation of scientific workforce who can straddle boundaries across multiple disciplines such as robotics, computer science, and ecology.This research aims to enable tighter integration of human-robot intelligence by teasing out components of efficient nonverbal human-robot communication. These include: (i) level of engagement of the robotic swarm as a function of human cognitive load, (ii) recruitment strategies used by humans as they team up with the robotic swarm to map a complex dynamic environment, (iii) perception of robot swarming patterns by humans, and (iv) indirect indicators of human cognitive load that can enable faster interpretation in the wild. Towards this, experimental conditions will highlight the dependence of team performance on how robots respond to the cognitive load experienced by the human participants. Experiments will be conducted in virtual reality to enable realization of large robot swarms without the accompanying design and sensor programming challenges. Scalability of swarm robotics will be preserved by building all environmental sensing and interaction strategies upon local interaction rules. Information-theoretic measures of directional information flow will be used to quantify human perception of swarm patterns and isolate movement correlates of cognitive load. This project has the support of the Human-Centered-Computing Program in the IIS Division in the CISE Directorate, and the NSF 2026 Fund Program in the Office of Integrated Activities. The project enriches, extends, and explores the NSF 2026 Idea Machine Winning Entry “Integrated Human-Machine Intelligence”.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.