CHS: Medium: Collaborative Research: Wearable Sound Sensing and Feedback Techniques for Persons who are Deaf or Hard of Hearing

本研究的目标是推进可穿戴声音传感和反馈技术的用户谁是聋人或重听，沿着适当的人机交互技术。 将开发具有多个麦克风和音频处理算法的可穿戴设备，以自动感知，定位和识别声音。 通过新兴的可穿戴技术（如头戴式显示器和智能手表）谨慎地向用户提供这些信息的方法也将被实施。 评估将包括实验室和实地研究，日常任务，如注意声音和参与口头对话，以及客观和主观的措施。项目成果将产生广泛的影响，为失聪或听力困难的用户提供新的声音感知选项，从而用额外的、不显眼的信息增强佩戴者的现有策略。新的辅助技术将有可能改善大部分人口的生活，特别是美国越来越多的老年听力损失患者。为此，主要的子目标将包括：了解用户对可穿戴声音感测和反馈的需求，包括在各种环境中并基于个体用户的听力损失水平对不同声音的重要性进行优先级排序;开发和评估轻型可穿戴声音传感平台和相关算法，包括用于麦克风阵列的新声音场景分析算法，该麦克风阵列共形于复杂形状的挡板上或附近（佩戴者的头部）并且考虑声音如何从佩戴者的身体散射，沿着自适应最先进的声音分类和语音识别方法来处理该处理的音频;通过可穿戴原型开发和评估感知声音的视觉和触觉或振动反馈。该奖项反映了NSF的法定使命，并被认为是值得的。通过使用基金会的知识价值和更广泛的影响审查标准进行评估来提供支持。

The goal of this research is to advance wearable sound sensing and feedback technology for users who are deaf or hard of hearing, along with appropriate techniques for human-computer interaction. A wearable device with multiple microphones and audio processing algorithms to automatically sense, localize, and identify sounds will be developed. Means to discreetly provide this information to the user via emerging wearable technologies such as head-mounted displays and smartwatches will also be implemented. Evaluations will include lab and field studies, everyday tasks such as noticing sounds and participating in oral conversations, and objective and subjective measures. Project outcomes will have broad impact by enabling new sound awareness options for users who are deaf or hard of hearing, thereby augmenting the wearer's existing strategies with additional, unobtrusive information. The new assistive technologies will have the potential to improve the lives of a large portion of the population, in particular the growing number of older adults with hearing loss in the United States.To these ends, major subgoals will include: understanding user needs for wearable sound sensing and feedback, including prioritizing the importance of different sounds across a variety of contexts and based on an individual user's level of hearing loss; developing and evaluating a lightweight wearable sound sensing platform and accompanying algorithms, including both new sound scene analysis algorithms for a microphone array conformal on, or in proximity to, a complex-shaped baffle (the wearer's head) and that take into account how sound scatters off the wearer's body, along with adaptive state-of-the-art sound classification and speech recognition approaches to work with this processed audio; and developing and evaluating visual and haptic or vibrational feedback of the sensed sound via wearable prototypes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.