CAREER: Learning from Interfaces for Simulation Environments to Design Wearable Technologies and Mixed-Reality Testbeds for Disaster Response Teams

职业：从模拟环境界面中学习，为灾难响应团队设计可穿戴技术和混合现实测试台

• 批准号: 1651532
• 负责人: Phoebe Toups Dugas
• 金额: $ 55万
• 依托单位: New Mexico State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1651532&HistoricalAwards=false
• 关键词: CAREER; Learning; Interfaces; Simulation; Environments

This project's goal is to support disaster response teams -- particularly urban search and rescue -- by designing wearable technologies that meet their information needs. Wearables for disaster response are promising because responders' hands are often busy and their awareness must be on their surroundings rather than a screen, making it hard to use common computing devices like smartphones, tablets, or computers. A number of prototype wearable technologies exist that might help: head-up displays can show data alongside the physical world; audio headsets can enable communication with both teammates and computers; armbands can provide touch-based feedback and allow gestures for communication and control; sensors can detect location, motion, and aspects of people's physical and mental state. However, usable wearable systems and design guidance for building them is scarce in general and even scarcer in the context of disaster response. The key insight behind the proposed work is that interfaces for training simulations and computer games make heavy use of head-up displays and specialized controllers; further, these often share elements of real disaster response scenarios. This project will study the features and effectiveness of these interfaces to generate design guidelines for wearable computing systems. Working closely with the Texas Task Force One response team, the project team will create custom-built wearable systems to support their mission as well as purpose-built mixed reality training simulations that combine virtual simulation with physical-world settings. The team will validate those simulations with the disaster response partners and use them to test and improve both the wearable technologies and the design guidelines the team creates. Both the work itself and the lessons learned will be used to improve classes at the lead investigator's institution around designing mixed reality technologies and human-computer interaction (HCI); the lead, and his institution, are also committed to broadening participation in computing education and computing research.The work will start with a deep study of training practices and needs for disaster response teams. The team will closely collaborate with its existing task force partners, using ethnographic observation of their existing training practices and interviews with task force leaders to develop models of task force training requirements and design considerations for wearable systems to support them. These will be disseminated both to disaster response teams and the HCI community. In parallel, the team will extract design best practices from existing training simulations and computer games, focusing on those that align with disaster response scenarios and needs. Team members will analyze the interaction techniques and mechanics these systems use -- how they provide situation awareness through ancillary displays, interface controls for managing large numbers of units and functions, and communication facilities for multiple participants -- along with people's preferences for and ability to use them. This will lead to a design catalog of existing interfaces and best practices for designing wearable and mixed reality interfaces that, along with the requirements identified through studying the task force, will inform the design of both wearable interfaces and testbed training simulations for disaster response contexts. These design activities will involve regular communication with task force partners to get feedback on using the interfaces in real situations, helping to align both the technology and simulation designs with real disaster response needs. Finally, the team will evaluate both the interfaces and the developed testbeds through a number of small-scale deployments and experiments, looking at how they support situation awareness, efficiency, and communication, as well as their effect on stress and workload. Much of the work will be used to support both courses and outreach activities; these include developing cases and projects based on the work for an HCI class as well as leveraging the designed technologies to improve a course on mixed reality. These will also be used to support a mixed reality interface development outreach activity with the local community college to expose students from underrepresented groups to both research and to work with advanced computing technology.

该项目的目标是通过设计满足其信息需求的可穿戴技术来支持灾害响应团队，特别是城市搜索和救援。 用于灾难响应的可穿戴设备很有前途，因为响应者的手通常很忙碌，他们的意识必须在周围而不是屏幕上，这使得很难使用智能手机，平板电脑或计算机等常见的计算设备。 一些原型可穿戴技术可能会有所帮助：平视显示器可以显示物理世界的数据;音频耳机可以实现与队友和计算机的通信;臂章可以提供基于触摸的反馈，并允许手势进行通信和控制;传感器可以检测位置，运动以及人们的身体和精神状态。 然而，可用的可穿戴系统和设计指导，以建立他们是稀缺的一般情况下，甚至在灾害响应的背景下更稀缺。 拟议工作背后的关键见解是，培训模拟和计算机游戏的界面大量使用平视显示器和专用控制器;此外，这些通常共享真实的灾难响应场景的元素。 这个项目将研究这些接口的功能和有效性，以生成可穿戴计算系统的设计指南。 该项目团队将与德克萨斯州特遣部队一号响应团队密切合作，创建定制的可穿戴系统，以支持他们的使命，以及将联合收割机虚拟仿真与物理世界设置相结合的专用混合现实训练模拟。 该团队将与灾难响应合作伙伴一起验证这些模拟，并使用它们来测试和改进可穿戴技术和团队创建的设计指南。 这项工作本身和所获得的经验教训都将用于改进首席研究员所在机构的混合现实技术和人机交互（HCI）设计课程;首席研究员及其所在机构还致力于扩大对计算教育和计算研究的参与。这项工作将从深入研究灾难响应团队的培训实践和需求开始。 该团队将与现有的工作队合作伙伴密切合作，利用对他们现有培训实践的人种学观察和对工作队领导人的采访，开发工作队培训需求模型和可穿戴系统的设计考虑因素，以支持他们。 这些文件将分发给救灾小组和人道主义行动社区。 与此同时，该团队将从现有的培训模拟和计算机游戏中提取设计最佳实践，重点关注那些与灾害响应场景和需求相一致的实践。 团队成员将分析这些系统使用的交互技术和机制-它们如何通过辅助显示器提供态势感知，界面控制用于管理大量单元和功能，以及多个参与者的通信设施-沿着人们的偏好和使用能力。 这将产生现有接口的设计目录和设计可穿戴和混合现实接口的最佳实践，这些接口以及通过研究工作组确定的要求，沿着将为可穿戴接口和测试台的设计提供信息。灾难响应环境的培训模拟。 这些设计活动将包括与工作队伙伴定期沟通，以获得在真实的情况下使用接口的反馈，帮助使技术和模拟设计与真实的救灾需要保持一致。 最后，该团队将通过一些小规模的部署和实验来评估接口和开发的测试平台，研究它们如何支持态势感知、效率和通信，以及它们对压力和工作负载的影响。 大部分工作将用于支持课程和推广活动;其中包括基于HCI类的工作开发案例和项目，以及利用设计的技术来改进混合现实课程。 这些还将用于支持与当地社区学院的混合现实界面开发推广活动，使来自代表性不足群体的学生既能从事研究，又能使用先进的计算技术。

项目成果

"It Started as a Joke": On the Design of Idle Games

《它开始是个玩笑》：闲置游戏的设计

• DOI: 10.1145/3311350.3347180
• 发表时间: 2019
• 期刊: CHI PLAY '19: Proceedings of the Annual Symposium on Computer-Human Interaction in Play
• 作者: Spiel, Katta;Alharthi, Sultan A.;Cen, Andrew Jian-lan;Hammer, Jessica;Nacke, Lennart E.;Toups Dugas, Phoebe O.;Tanenbaum, Theresa Jean
• 通讯作者: Tanenbaum, Theresa Jean

A Vision of Augmented Reality for Urban Search and Rescue

• DOI: 10.1145/3363384.3363466
• 发表时间: 2019-11
• 期刊: Proceedings of the Halfway to the Future Symposium 2019
• 作者: Nicolas J. Lalone;Sultan A. Alharthi;Z. Toups

A Quest?!: The Secret Life of Gameworld Punctuation

任务？！：游戏世界标点符号的秘密生活

• DOI: 10.1145/3611040
• 发表时间: 2023
• 期刊: Proceedings of the ACM on Human-Computer Interaction
• 作者: LaLone, Nicolas;Toups Dugas, Phoebe O.;Cormier, Michelle V.
• 通讯作者: Cormier, Michelle V.

Investigating the Impact of Annotation Interfaces on Player Performance in Distributed Multiplayer Games

研究注释接口对分布式多人游戏中玩家表现的影响

• DOI: 10.1145/3173574.3173888
• 发表时间: 2018
• 期刊: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
• 作者: Alharthi, Sultan A.;Torres, Ruth C.;Khalaf, Ahmed S.;Toups, Zachary O.;Dolgov, Igor;Nacke, Lennart E.
• 通讯作者: Nacke, Lennart E.

A Design Framework for Awareness Cues in Distributed Multiplayer Games

分布式多人游戏中意识线索的设计框架

• DOI: 10.1145/3173574.3173817
• 发表时间: 2018
• 期刊: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
• 作者: Wuertz, Jason;Alharthi, Sultan A.;Hamilton, William A.;Bateman, Scott;Gutwin, Carl;Tang, Anthony;Toups, Zachary;Hammer, Jessica
• 通讯作者: Hammer, Jessica