CHS: Medium: Collaborative Research: Wearable Sound Sensing and Feedback Techniques for Persons who are Deaf or Hard of Hearing

本研究的目标是为失聪或重听用户推进可穿戴式声音传感和反馈技术，以及适当的人机交互技术。将开发具有多个麦克风和音频处理算法的可穿戴设备，以自动感知、定位和识别声音。通过头戴式显示器和智能手表等新兴可穿戴技术，将这些信息谨慎地提供给用户的方法也将得到实施。评估将包括实验室和实地研究，日常任务，如注意声音和参与口头对话，以及客观和主观的衡量。项目成果将产生广泛的影响，为失聪或重听用户提供新的声音感知选项，从而用额外的、不引人注目的信息增强佩戴者的现有策略。新的辅助技术将有潜力改善大部分人口的生活，特别是在美国越来越多的老年人听力损失。为此，主要子目标将包括：了解用户对可穿戴声音传感和反馈的需求，包括根据个人用户的听力损失水平，在各种情况下优先考虑不同声音的重要性；开发和评估轻量级可穿戴声音传感平台和伴随算法，包括用于在复杂形状的隔板（佩戴者的头部）上或附近的麦克风阵列的新声音场景分析算法，并考虑到声音如何从佩戴者的身体散射，以及自适应的最先进的声音分类和语音识别方法，以处理此处理过的音频；通过可穿戴原型开发和评估感知声音的视觉、触觉或振动反馈。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The goal of this research is to advance wearable sound sensing and feedback technology for users who are deaf or hard of hearing, along with appropriate techniques for human-computer interaction. A wearable device with multiple microphones and audio processing algorithms to automatically sense, localize, and identify sounds will be developed. Means to discreetly provide this information to the user via emerging wearable technologies such as head-mounted displays and smartwatches will also be implemented. Evaluations will include lab and field studies, everyday tasks such as noticing sounds and participating in oral conversations, and objective and subjective measures. Project outcomes will have broad impact by enabling new sound awareness options for users who are deaf or hard of hearing, thereby augmenting the wearer's existing strategies with additional, unobtrusive information. The new assistive technologies will have the potential to improve the lives of a large portion of the population, in particular the growing number of older adults with hearing loss in the United States.To these ends, major subgoals will include: understanding user needs for wearable sound sensing and feedback, including prioritizing the importance of different sounds across a variety of contexts and based on an individual user's level of hearing loss; developing and evaluating a lightweight wearable sound sensing platform and accompanying algorithms, including both new sound scene analysis algorithms for a microphone array conformal on, or in proximity to, a complex-shaped baffle (the wearer's head) and that take into account how sound scatters off the wearer's body, along with adaptive state-of-the-art sound classification and speech recognition approaches to work with this processed audio; and developing and evaluating visual and haptic or vibrational feedback of the sensed sound via wearable prototypes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.