Towards more inclusive speech technologies: speech enhancement in atypical conditions

迈向更具包容性的语音技术：非典型条件下的语音增强

• 批准号: RGPIN-2021-03182
• 负责人: Bouserhal, Rachel
• 金额: $ 2.04万
• 依托单位: École de technologie supérieure
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=742424
• 关键词: Towards; more; inclusive; speech; technologies

In an aging population that is increasingly affected by hearing impairment where intelligent devices such as home assistants and wearable in-ear devices rely on speech for human-human as well as human-machine communication, changes in speech production must be assessed and well understood to optimize the performance of such devices for all talkers. Current speech technologies, including speech recognition systems, factor in disturbances on speech production such as noise, or reverberant environments, but they fail to take into account other common disturbances.  In fact, current speech technologies are trained on open--ear normal-hearing talkers and do not include the increasing population of hearing impaired talkers. Furthermore, occluding the earcanal with an intra--aural device, such as a hearing protection device or an earphone, causes changes to speech production that are detrimental to speech intelligibility, especially in noise. In addition, obstruction of the mouth with a face mask or respirator degrades speech intelligibility and the performance of these tools, which is especially relevant in the current pandemic. The long-term goal of this research program is to make speech technologies more inclusive by developing tools that are suitable and adaptable to hearing impairments, including age-induced and noise-induced hearing loss, and speech degredations, such as those caused by the use of face masks and respirators. This will be achieved by modeling changes in speech production caused by a number of disturbances, and subsequently developing speech enhancement algorithms that amend the degredations on speech intelligibility caused by these changes. Two large databases of speech sentences produced in atypical conditions will be collected and made available through open-access. To understand the changes in speech production caused by each disturbance, linear mixed models will be created for acoustic features that are tied to intelligibility such as intensity, spectral tilt, vowel duration, and formant bandwidth.  Subsequently, speech enhancement algorithms such as spectral envelope mapping, bandwidth extension, and near-end listening enhancement techniques will be employed and adapted to work in real time on low capacity portable devices in noisy environments. The outcomes of this research program will have impact on the general population, the scientific community, and the industrial sector, in Canada and other parts of the world. They will benefit millions of Canadians suffering from hearing impairment and reduce work related stress for medical staff that use PPEs that obstruct the mouth.

在日益受到听力障碍影响的老龄化人口中，智能设备（如家庭助理和可穿戴入耳式设备）依赖语音进行人与人以及人机通信，必须评估并充分理解语音产生的变化，以优化此类设备对所有说话者的性能。当前的语音技术，包括语音识别系统，考虑了对语音产生的干扰，如噪声或混响环境，但它们没有考虑到其他常见的干扰。事实上，当前的语音技术是在开耳听力正常的说话者上训练的，并且不包括日益增长的听力受损的说话者。此外，用耳内装置（例如听力保护装置或耳机）阻塞耳道会导致语音产生的变化，这对语音清晰度（尤其是噪声）有害。此外，用面罩或呼吸器阻塞口腔会降低语言清晰度和这些工具的性能，这在当前的大流行中尤其相关。 该研究计划的长期目标是通过开发适合和适应听力损伤的工具，使语音技术更具包容性，包括年龄引起的和噪音引起的听力损失，以及使用面罩和呼吸器引起的语音退化。这将通过模拟由一些干扰引起的语音产生变化来实现，并且随后开发语音增强算法，该语音增强算法修正由这些变化引起的语音可懂度的等级。将收集非典型条件下产生的言语句子的两个大型数据库，并通过开放存取提供。为了理解每种干扰引起的语音产生的变化，将为与可懂度相关的声学特征（如强度、频谱倾斜、元音持续时间和共振峰带宽）创建线性混合模型。随后，语音增强算法（如频谱包络映射、带宽扩展、并且近端收听增强技术将被采用并适于在噪声环境中的低容量便携式设备上真实的工作。这项研究计划的成果将对加拿大和世界其他地区的普通民众、科学界和工业部门产生影响。他们将使数百万患有听力障碍的加拿大人受益，并减轻使用PPE阻塞口腔的医务人员的工作压力。