EAGER: Improved Situation Awareness of Unknown Environments through a Robotic Augmented Reality Virtual Window

EAGER：通过机器人增强现实虚拟窗口提高对未知环境的态势感知

• 批准号: 1937565
• 负责人: John Swan
• 金额: $ 15万
• 依托单位: Mississippi State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1937565&HistoricalAwards=false
• 关键词: EAGER; Improved; Situation; Awareness; Unknown

When responding to a search warrant or 911 call, teams of police officers often have to enter and search a room. The problem the police officers face is that the contents of the room are unknown and might be dangerous. Therefore, the activity is stressful, and sometimes results in adverse outcomes. This project addresses this problem by developing a robot that police officers can send into a room before they enter. The robot uses cameras and other sensors to see the room, even in the dark. The police officers wear augmented reality headsets that allow them to see the room through the robot's cameras. In addition, the combination of augmented reality and the robot allows the police to see the room as though they had x-ray vision, giving them a virtual window through an otherwise solid wall. Seeing the room through this virtual window would make searching the room more efficient, reduce uncertainty and stress, and result in avoided injuries and saved lives. More broadly, the project will further the science of using robots and augmented reality with skilled teams performing dangerous work. The lessons learned could advance efforts to incorporate robots into related activities, including additional policing tasks, firefighting, inspecting buildings and warehouses, responding to alarm activation, searching and rescuing, and military operations.The objectives of this project are (1) to explore the use of an augmented reality virtual window to provide x-ray vision, where otherwise invisible objects can be seen, and (2) to use a robot to locate and identify objects in unknown, potentially dangerous, and stressful environments. These objectives are pursued in the context of a common task -- the room-clearing scenario -- undertaken by teams of police officers. This project will develop a non-weaponized robot, which team members will send into a simulated apartment with several rooms. The robot will view each room from robot-mounted cameras and sensors, and will present the view in an augmented reality (AR) virtual window. The cameras and sensors will see the room in multiple spectra, including visible, thermal, and laser, which will allow viewing in varying levels of illumination, including darkness. The project will leverage the investigators' longstanding relationship with regional police SWAT teams (Special Weapons And Tactics), who have agreed to provide personnel to be subject matter experts, design partners, and evaluation subjects. The project will be conducted in an existing tactical robotic testbed facility, and spatial understanding and situation awareness will be measured, using both standard and custom methods and surveys. The work has the potential to advance the fields of augmented reality, robotics, and human-robot interaction, as well as to advance the day when robots can be successfully integrated into SWAT teams.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当回应搜查令或911电话时，警察团队经常不得不进入并搜查一个房间。 警察面临的问题是，房间里的东西是未知的，可能是危险的。 因此，活动是紧张的，有时会导致不良后果。 该项目通过开发一种机器人来解决这个问题，警察可以在进入房间之前将其送入房间。 机器人使用摄像头和其他传感器来查看房间，即使在黑暗中。 警察戴着增强现实耳机，通过机器人的摄像头看到房间。 此外，增强现实和机器人的结合使警察能够看到房间，就像他们有X射线视觉一样，给他们一个虚拟的窗户，通过一个坚固的墙壁。 通过这个虚拟窗口看到房间将使搜索房间更有效，减少不确定性和压力，从而避免伤害并挽救生命。 更广泛地说，该项目将进一步推动使用机器人和增强现实技术的科学，让熟练的团队从事危险的工作。 这些经验教训可以促进将机器人纳入相关活动的努力，包括额外的警务任务，消防，检查建筑物和仓库，响应警报激活，搜索和救援以及军事行动。本项目的目标是（1）探索使用增强现实虚拟窗口提供X射线视觉，在那里可以看到看不到的物体，以及（2）使用机器人在未知的、潜在危险的和有压力的环境中定位和识别物体。 这些目标是在警察小组执行的一项共同任务-清理房间设想-的范围内实现的。 这个项目将开发一个非武器化的机器人，团队成员将把它送到一个有几个房间的模拟公寓里。 机器人将通过安装在机器人上的摄像头和传感器查看每个房间，并将在增强现实（AR）虚拟窗口中呈现视图。 摄像头和传感器将在多个光谱中看到房间，包括可见光，热和激光，这将允许在不同的照明水平下观看，包括黑暗。 该项目将利用调查人员与区域警察特警队（特种武器和战术）的长期关系，这些特警队已同意提供人员担任主题专家、设计合作伙伴和评估对象。 该项目将在现有的战术机器人试验台设施中进行，并将使用标准和定制方法和调查来测量空间理解和态势感知。 这项工作有可能推动增强现实、机器人技术和人机交互领域的发展，并推动机器人成功融入特警队的那一天。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

A Robotic Augmented Reality Virtual Window for Law Enforcement Operations

• DOI: 10.1007/978-3-030-49695-1_40
• 发表时间: 2020-07
• 作者: Nate Phillips;Brady Kruse;Farzana Alam Khan;J. Edward Swan II;Cindy L. Bethel

The Effect of Context Switching, Focal Switching Distance, Binocular and Monocular Viewing, and Transient Focal Blur on Human Performance in Optical See-Through Augmented Reality

• DOI: 10.1109/tvcg.2022.3150503
• 发表时间: 2022-05-01
• 期刊: IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
• 影响因子: 5.2
• 作者: Arefin, Mohammed Safayet;Phillips, Nate;Swan, J. Edward, II
• 通讯作者: Swan, J. Edward, II

An X-Ray Vision System for Situation Awareness in Action Space

用于行动空间态势感知的 X 射线视觉系统

• DOI: 10.1109/vrw52623.2021.00179
• 发表时间: 2021
• 期刊: IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW 2021
• 作者: Phillips, Nate;Khan, Farzana Alam;Kruse, Brady;Bethel, Cindy;Swan II, J. Edward
• 通讯作者: Swan II, J. Edward

Effects of a Distracting Background and Focal Switching Distance in an Augmented Reality System

增强现实系统中分散注意力的背景和焦点切换距离的影响

• DOI: 10.1109/ismar-adjunct54149.2021.00029
• 发表时间: 2021
• 期刊: IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct
• 作者: Arefin, Mohammed Safayet;Phillips, Nate;Plopski, Alexander;Swan II, J. Edward
• 通讯作者: Swan II, J. Edward

A Method for Measuring the Perceived Location of Virtual Content in Optical See Through Augmented Reality

一种测量光学透视增强现实中虚拟内容感知位置的方法

• DOI: 10.1109/vrw52623.2021.00211
• 发表时间: 2021
• 期刊: IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW
• 作者: Khan, Farzana Alam;Muvva, Veera Venkata;Wu, Dennis;Arefin, Mohammed Safayet;Phillips, Nate;Swan, J. Edward
• 通讯作者: Swan, J. Edward