HCC-Small: Assistive Listening Devices and Voice Processing Platforms for the Deaf and Hard of Hearing

本研究旨在开发辅助听力装置和语音处理平台，以促进与聋人和重听人的交流。为了实现这一目标，PI和他的团队将采用现有的语音处理和模式识别技术，并为全新的人机交互形式奠定基础。将开发的具体设备和平台将从以下五个类别中抽取。(1)对话翻译器，方便聋人参与双语或多语对话；假设对话发生在一张桌子上（在家里或办公室），桌子上有用于语音识别的cpu和用于显示的监视器。(2)一种声音监测器，为听力困难的用户提供关于他们声音投射水平的即时、微妙的反馈；如果没有这种反馈，不适当的调节会在社交互动中引入尴尬，随着时间的推移导致退缩和孤立。(3)语音罗盘，可在多人对话时提供快速方向通知；这种设备将帮助聋人读唇，否则他们无法追踪不同说话者的轮流（在这种情况下，读唇假定有能力知道该往哪里看）。(4)音频事件检测器，识别用户环境中的显著词语、提示和声音；这样的设备可以让听力受损的用户听到有人叫他们的名字或与工作有关的重要声音，从而减少挫败感，提高就业能力。(5)使用原始语音识别技术的可穿戴式收发器，允许有语音和听力的用户与失聪晚期的爱人交流简单的文字；这种装置将补充残余的听力、手势和唇读。更广泛的影响：超过3500万美国人有某种形式的听力损失。此外，估计这些病例中有五分之一涉及严重或深度听力损失。听力损失的发病率随着年龄的增长而增加：大约2%的18岁以下儿童受到影响，但超过30%的65岁以上成年人受到影响。因此，受影响的成年人的比例将随着婴儿潮一代的老龄化而增长。对于听力损失患者在家庭、社区和工作场所更好地融入社会有着巨大的需求。聋人社区成员与听力正常人群之间经常存在着严重的沟通障碍。即使在一个家庭中，这些障碍也会给失聪的家庭成员带来绝望的孤立感。在工作场所，聋人社区的成员和其他人一样健全，但沟通上的困难造成了最佳就业和保留现有职位的障碍。

This research aims to develop assistive listening devices and voice processing platforms that facilitate communication with the deaf and hard of hearing. To achieve this goal, the PI and his team will both adapt existing technologies in speech processing and pattern recognition, and lay the foundation for entirely novel forms of human-computer interaction. Specific devices and platforms that will be developed will be drawn from the following five categories. (1) A dialogue interpreter that facilitates the inclusion of deaf users in dual or multi-speaker conversations; the conversations are assumed to take place at a table (in the home or office), which accommodates CPUs for speech recognition and monitors for display. (2) A voice monitor that provides hard-of-hearing users with immediate, subtle feedback on the level of their vocal projections; without this feedback, inappropriate modulation can introduce awkwardness in social interactions, leading over time to withdrawal and isolation. (3) A voice compass that provides rapid directional notification during multi-speaker conversation; such a device would assist deaf lip-readers who cannot otherwise track the turns of different speakers (lip reading in this setting presumes the ability to know where to look). (4) An audio event detector that recognizes salient words, prompts, and sounds in the user's environment; such a device could notify users with hearing loss to someone calling their name or to important work-related sounds, thus reducing frustrations and boosting employability. (5) A wearable transmitter/receiver that uses primitive speech recognition to allow a user with speech and hearing to communicate simple words to a late deafened love one; such a device would complement residual hearing, gestures, and lip-reading.Broader Impacts: Over 35 million Americans have some form of hearing loss. Moreover, one-fifth of these cases are estimated to involve severe or profound hearing loss. Incidence of hearing loss increases with age: it affects roughly 2% of children under age 18 but over 30% of adults over age 65. Thus, the percentage of affected adults will grow with the aging of the baby-boomer generation. There is a tremendous need for better integration of people with hearing loss in the home, community, and workplace. Extreme barriers in communication often exist between members of the Deaf community and the hearing population. Even within a single household, these barriers can create a desperate sense of isolation for a family member who is deaf. In the workplace, the members of the Deaf community are as able-bodied as the rest of the population, but difficulties in communication create barriers to optimal employment and retention in current positions.