Real-time deep learning to improve speech intelligibility in noise

项目摘要/摘要 每八个美国人中就有一个患有听力损失，这构成了重大的健康和经济负担 (Blackwell等人，2014)。听力受损(HI)听众的主要抱怨是理解困难 当存在背景噪音时的语音(见Dillon，2012)。虽然助听器(HAS)在 近年来，它们在嘈杂的环境中仍然提供的好处仍然很少。几十年来，一种改善 在背景噪音中理解语音的能力似乎是无法实现的，尽管有大量的 由大学和医管局公司进行的研究。当深度学习提供了第一个 演示了单麦克风算法，该算法可明显改善HI听众的噪声(Healy等人， 2013、2014、2015)。尽管该算法提供了大量的可理解性改进(甚至允许 以提高从地板到天花板的清晰度)，但目前还没有真正实现 因此不适合在人工耳机和人工耳蜗术(CI)中实施。我们需要的是， 因此，是一种能够实时运行的高效降噪算法。这个项目 旨在满足这一迫切需求。 目前提议的项目的长期目标是缓解HI听众的主要听力 障碍，即在背景噪音中理解语音的困难。第一个目标是引入一个新的 基于一种新的基础方案的算法，旨在为任何HI提供实质性好处 实时监听。此算法将非常适合于在HAS、CI和其他面对面的应用中实施 通信应用程序。这一新算法的有效性将使用HI和 正常听力(NH)听众。第二个目标是对这个新算法进行扩展，将其修改为接受 少量的未来时间帧信息，这可能会改善其降噪性能，但将 引入短暂的处理延迟。其基本原理是不同的设备具有不同的允许延迟。 面对面通信设备(HAS、CI等)具有严格的低延迟要求，但其他重要的 通信系统(例如，电话)具有不同的要求。未来的增加是有可能的 在这些要求内的时间帧信息(最多150毫秒)将导致更好的语音清晰度。 但任何潜在好处的规模尚不清楚。这一关键信息将在当前确定。 使用HI和NH收听者，我们将测量已处理的嘈杂句子的可理解性 使用不同数量的未来时间信息。 这项全面的奖学金培训计划将提供个性化的、有指导的研究培训 在高度支持和高效的环境中拥有世界一流的教师队伍。拟议的工作将赋予 申请人具备过渡到研究事业下一阶段所需的技能，改变了我们对 听力损失，并对数百万美国人的生活质量产生重大影响。

Project Summary/Abstract One in eight Americans has hearing loss, and this constitutes a major health and economic burden (Blackwell et al., 2014). The primary complaint of hearing-impaired (HI) listeners is difficulty understanding speech when background noise is present (see Dillon, 2012). While hearing aids (HAs) have improved in recent years, they still provide little benefit in noisy environments. For decades, a means of improving the ability to understand speech in background noise appeared unattainable, despite substantial amounts of research by both universities and HA companies. This changed when deep learning provided the first demonstration of a single-microphone algorithm that improves intelligibly in noise for HI listeners (Healy et al., 2013, 2014, 2015). Although this algorithm provides massive intelligibility improvements (even allowing listeners to improve intelligibility from floor to ceiling levels), it is currently not implemented to operate in real time and is therefore not suitable for implementation into HAs and cochlear implants (CIs). What is needed, therefore, is a highly effective noise-reduction algorithm that is capable of operating in real time. This project aims to address this critical need. The long-term goal of the currently proposed project is to alleviate HI listeners’ predominant hearing handicap, which is difficulty understanding speech in background noise. The first aim introduces a new algorithm, based on a novel foundational scheme, that is designed to provide substantial benefit for any HI listener in real time. This algorithm will be well suited for implementation into HAs, CIs, and other face-to-face communication applications. The effectiveness of this new algorithm will be quantified using both HI and normal-hearing (NH) listeners. The second aim expands upon this new algorithm by modifying it to accept a small amount of future time-frame information, which could improve its noise-reduction performance but will introduce a brief processing delay. The rationale is that different devices have different allowable latencies. Face-to-face communication devices (HAs, CIs, etc.) have strict low-latency requirements, but other important communication systems (e.g., telephones) have different requirements. It is possible that the addition of future time-frame information within these requirements (up to 150 ms) will result in even better speech intelligibility. But the magnitude of any potential benefit is unknown. This critical information will be established currently. Using both HI and NH listeners, we will measure intelligibility for noisy sentences that have been processed using various amounts of future time information. This comprehensive fellowship training plan will provide individualized, mentored research training from world-class faculty in a highly supportive and productive environment. The proposed work will endow the applicant with the skills needed to transition to the next stage of his research career, transform our treatment of hearing loss, and substantially impact quality of life for millions of Americans.