EAGER: Improved Situation Awareness of Unknown Environments through a Robotic Augmented Reality Virtual Window

在接到搜查令或911报警电话时，一队警察经常要进入房间进行搜查。警察面临的问题是，房间里的东西是未知的，可能很危险。因此，这种活动是有压力的，有时会导致不良后果。这个项目通过开发一种机器人来解决这个问题，警察可以在他们进入房间之前把它送进去。即使在黑暗中，机器人也可以使用摄像头和其他传感器来观察房间。警察戴着增强现实头盔，可以通过机器人的摄像头看到房间里的情况。此外，增强现实和机器人的结合使警察可以像拥有x光一样看到房间，为他们提供了一扇虚拟的窗户，透过一堵坚固的墙。通过这个虚拟窗口看到房间将使搜索房间更有效率，减少不确定性和压力，并避免伤害和挽救生命。更广泛地说，该项目将进一步推动使用机器人和增强现实技术的科学，让熟练的团队执行危险的工作。吸取的经验教训可以推动将机器人纳入相关活动的努力，包括额外的警务任务、消防、检查建筑物和仓库、响应警报激活、搜索和救援以及军事行动。该项目的目标是：(1)探索使用增强现实虚拟窗口来提供x射线视觉，在那里可以看到原本看不见的物体；(2)使用机器人在未知的、潜在危险的和有压力的环境中定位和识别物体。这些目标是在一项共同任务的背景下实现的，即由警察小组承担的房间清理任务。这个项目将开发一个非武器机器人，团队成员将把它送入一个有几个房间的模拟公寓。机器人将通过安装在机器人上的摄像头和传感器查看每个房间，并将在增强现实（AR）虚拟窗口中呈现视图。摄像机和传感器将在多种光谱下观察房间，包括可见光、热光谱和激光光谱，这将允许在不同的照明水平下观察，包括黑暗。该项目将利用调查人员与地区警察SWAT小组（特种武器和战术）的长期关系，后者已同意提供人员作为主题专家、设计合作伙伴和评估对象。该项目将在现有的战术机器人测试平台设施中进行，并将使用标准和定制方法和调查来测量空间理解和态势感知。这项工作有可能推动增强现实、机器人技术和人机交互领域的发展，也有可能推动机器人成功集成到SWAT团队中的那一天。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

When responding to a search warrant or 911 call, teams of police officers often have to enter and search a room. The problem the police officers face is that the contents of the room are unknown and might be dangerous. Therefore, the activity is stressful, and sometimes results in adverse outcomes. This project addresses this problem by developing a robot that police officers can send into a room before they enter. The robot uses cameras and other sensors to see the room, even in the dark. The police officers wear augmented reality headsets that allow them to see the room through the robot's cameras. In addition, the combination of augmented reality and the robot allows the police to see the room as though they had x-ray vision, giving them a virtual window through an otherwise solid wall. Seeing the room through this virtual window would make searching the room more efficient, reduce uncertainty and stress, and result in avoided injuries and saved lives. More broadly, the project will further the science of using robots and augmented reality with skilled teams performing dangerous work. The lessons learned could advance efforts to incorporate robots into related activities, including additional policing tasks, firefighting, inspecting buildings and warehouses, responding to alarm activation, searching and rescuing, and military operations.The objectives of this project are (1) to explore the use of an augmented reality virtual window to provide x-ray vision, where otherwise invisible objects can be seen, and (2) to use a robot to locate and identify objects in unknown, potentially dangerous, and stressful environments. These objectives are pursued in the context of a common task -- the room-clearing scenario -- undertaken by teams of police officers. This project will develop a non-weaponized robot, which team members will send into a simulated apartment with several rooms. The robot will view each room from robot-mounted cameras and sensors, and will present the view in an augmented reality (AR) virtual window. The cameras and sensors will see the room in multiple spectra, including visible, thermal, and laser, which will allow viewing in varying levels of illumination, including darkness. The project will leverage the investigators' longstanding relationship with regional police SWAT teams (Special Weapons And Tactics), who have agreed to provide personnel to be subject matter experts, design partners, and evaluation subjects. The project will be conducted in an existing tactical robotic testbed facility, and spatial understanding and situation awareness will be measured, using both standard and custom methods and surveys. The work has the potential to advance the fields of augmented reality, robotics, and human-robot interaction, as well as to advance the day when robots can be successfully integrated into SWAT teams.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.