CHS: Medium: Collaborative Research: Wearable Sound Sensing and Feedback Techniques for Persons who are Deaf or Hard of Hearing

这项研究的目标是为聋哑人或重听人推进可穿戴式声音感知和反馈技术，以及适当的人机交互技术。将开发一种具有多个麦克风和音频处理算法的可穿戴设备，以自动感知、定位和识别声音。还将实施通过头盔显示器和智能手表等新兴可穿戴技术将这些信息谨慎地提供给用户的手段。评估将包括实验室和实地研究，日常任务，如注意声音和参与口头对话，以及客观和主观测量。项目成果将产生广泛的影响，为聋人或重听用户提供新的声音感知选项，从而为佩戴者的现有策略增加额外的、不引人注目的信息。新的辅助技术将有可能改善很大一部分人的生活，特别是美国越来越多的老年人听力损失。为此，主要的子目标将包括：了解用户对可穿戴声音感知和反馈的需求，包括根据个人用户的听力损失水平，确定不同声音在各种背景下的重要性；开发和评估轻量级可穿戴声音传感平台和相关算法，包括用于复杂形状挡板(佩戴者头部)上或附近的麦克风阵列的新声音场景分析算法，以及考虑声音如何从佩戴者身体传播的算法，以及用于处理这些处理后的音频的自适应最先进的声音分类和语音识别方法；以及开发和评估通过可穿戴原型感知的声音的视觉和触觉或振动反馈。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The goal of this research is to advance wearable sound sensing and feedback technology for users who are deaf or hard of hearing, along with appropriate techniques for human-computer interaction. A wearable device with multiple microphones and audio processing algorithms to automatically sense, localize, and identify sounds will be developed. Means to discreetly provide this information to the user via emerging wearable technologies such as head-mounted displays and smartwatches will also be implemented. Evaluations will include lab and field studies, everyday tasks such as noticing sounds and participating in oral conversations, and objective and subjective measures. Project outcomes will have broad impact by enabling new sound awareness options for users who are deaf or hard of hearing, thereby augmenting the wearer's existing strategies with additional, unobtrusive information. The new assistive technologies will have the potential to improve the lives of a large portion of the population, in particular the growing number of older adults with hearing loss in the United States.To these ends, major subgoals will include: understanding user needs for wearable sound sensing and feedback, including prioritizing the importance of different sounds across a variety of contexts and based on an individual user's level of hearing loss; developing and evaluating a lightweight wearable sound sensing platform and accompanying algorithms, including both new sound scene analysis algorithms for a microphone array conformal on, or in proximity to, a complex-shaped baffle (the wearer's head) and that take into account how sound scatters off the wearer's body, along with adaptive state-of-the-art sound classification and speech recognition approaches to work with this processed audio; and developing and evaluating visual and haptic or vibrational feedback of the sensed sound via wearable prototypes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.