CHS: Small: Collaborative Research: 3D Audio Augmentation for Limited Field of View Augmented Reality Systems for Medical Training

CHS：小型：协作研究：用于有限视场的 3D 音频增强医疗培训增强现实系统

• 批准号: 1718313
• 负责人: Henry Fuchs
• 金额: $ 20万
• 依托单位: University of North Carolina at Chapel Hill
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2021-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1718313&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; 3D

Medical task simulators can provide a safe, affordable, and repeatable environment in which practitioners can rehearse procedures without impacting patient safety. Augmented reality (AR) is therefore the ideal display technology in this field, allowing the user to directly interact and practice skills within a natural environment. But current AR displays typically have a narrow field of view (FOV), which makes it difficult for users to immediately attend to an object outside of their periphery. This research will employ 3D audio to overcome that challenge. As a benefit to other AR and medical researchers, the system itself and other materials created for the present work (design, implementation, and tutorials) will be openly available on the project's website for all to use, modify, and contribute, and an open-source community will be created to link users and researchers who have similar interests as the present work. Undergraduate, women, and minority students will be engaged in all project activities through the Distributed Research Experiences for Undergraduates (DREU) program of the NSF-funded iAAMCS (Institute for African-American Mentoring in Computing Sciences), as well as the PI's courses on 3D sound design. Results will also be disseminated at relevant conference venues. The present work will advance research relating to the use of 3D audio for cueing in narrow FOV contexts, thereby improving interaction in large AR environments, and will create generalizable best-practices that lead to successful experiences when using AR devices. 
This research will develop the tools, methods, and infrastructure to evaluate how 3D sound can be used to enhance AR. This area of research is increasingly important, because as virtual reality (VR) and AR systems become more commonplace, it is imperative to design tools to overcome device limitations. The practical application of the proposed work will be realized through the evaluation of an AR-based prostate biopsy training procedure that will capture and reconstruct a full surgical procedure, with at least 3 dynamic participants in very close proximity. The project will pursue three main themes: (1) Infrastructure Development - The PIs and team will develop an extensible open-source software system that allows users to test their desired sound mappings; (2) Sound Mapping Quantification - Although sound has been used to convey spatial information in numerous contexts, the appropriate strategy for mapping periphery information to spatial sound attributes has not yet been determined, so the PIs propose a series of user studies to determine the mappings that most successfully help a user to attend to a target outside of their FOV; (3) Practical Application - To assess how well the proposed solution mitigates the challenge of a narrow FOV, a user study will be conducted to determine how the addition of 3D audio affects participants learning to perform the steps of a prostate biopsy procedure.

医疗任务模拟器可以提供一个安全、负担得起和可重复的环境，从业者可以在其中排练程序，而不会影响患者的安全。因此，增强现实(AR)是该领域的理想显示技术，允许用户在自然环境中直接交互和练习技能。但目前的AR显示器通常具有较窄的视场(FOV)，这使得用户很难立即注意到他们外围之外的对象。这项研究将使用3D音频来克服这一挑战。作为对其他AR和医学研究人员的好处，该系统本身和为本工作(设计、实施和教程)创建的其他材料将在项目网站上公开提供，供所有人使用、修改和贡献，并将创建一个开放源码社区，将与本工作有相似兴趣的用户和研究人员联系起来。本科生、女性和少数族裔学生将通过NSF资助的iAAMCS(非裔美国人计算科学指导研究所)的DREU计划以及PI的3D音效设计课程参与所有项目活动。会议结果还将在相关会议场所公布。本工作将推进与在窄视野环境中使用3D音频进行提示相关的研究，从而改善大型AR环境中的交互，并将创建可推广的最佳实践，从而在使用AR设备时获得成功经验。&amp；#8232；这项研究将开发工具、方法和基础设施来评估3D声音如何用于增强AR。这一领域的研究越来越重要，因为随着虚拟现实(VR)和AR系统变得越来越常见，设计工具来克服设备限制是势在必行的。拟议工作的实际应用将通过评估一种基于AR的前列腺活检培训程序来实现，该程序将捕获并重建一个完整的外科程序，其中至少有3名动态参与者非常接近。该项目将追求三个主要主题：(1)基础设施开发-声音映射量化--尽管声音已被用于在多种背景下传递空间信息，但将外围信息映射到空间声音属性的适当策略尚未确定，因此，私人投资者提议进行一系列用户研究，以确定最成功地帮助用户关注其视场之外的(3)实际应用-为了评估建议的解决方案在多大程度上缓解了狭窄视野的挑战，将进行一项用户研究，以确定添加3D音频对学习执行前列腺活检程序步骤的参与者有何影响。

项目成果

Mobile. Egocentric Human Body Motion Reconstruction Using Only Eyeglasses-mounted Cameras and a Few Body-worn Inertial Sensors

• DOI: 10.1109/vr50410.2021.00087
• 发表时间: 2021-03
• 期刊: 2021 IEEE Virtual Reality and 3D User Interfaces (VR)
• 作者: Young-Woon Cha;Husam Shaik;Qian Zhang;Fan Feng;A. State;A. Ilie;H. Fuchs