Towards more inclusive speech technologies: speech enhancement in atypical conditions

在日益受到听力障碍影响的老龄化人口中，家庭助理和可穿戴式入耳设备等智能设备依赖于人与人之间以及人机交流的语音，必须评估和充分了解语音产生的变化，以优化所有说话者的这些设备的性能。当前的语音技术，包括语音识别系统，考虑了语音产生的干扰因素，如噪声或混响环境，但它们没有考虑到其他常见的干扰。事实上，目前的语言技术是针对正常听力的人进行训练的，并没有包括越来越多的听力受损的人。此外，用耳内装置（如听力保护装置或耳机）堵塞耳道，会导致语音产生的变化，不利于语音的清晰度，尤其是在噪音中。此外，用口罩或呼吸器阻塞口腔会降低语音清晰度和这些工具的性能，这在当前的大流行中尤为重要。这项研究计划的长期目标是通过开发适合和适应听力障碍的工具，使语音技术更具包容性，包括年龄引起的和噪音引起的听力损失，以及使用口罩和呼吸器引起的语言退化。这将通过建模由许多干扰引起的语音产生的变化来实现，并随后开发语音增强算法来修正由这些变化引起的语音可理解性下降。将收集在非典型条件下产生的两个大型语音句子数据库，并通过开放访问提供。为了理解每种干扰引起的语音产生的变化，将为与可理解性相关的声学特征（如强度、频谱倾斜、元音持续时间和形成峰带宽）创建线性混合模型。随后，频谱包络映射、带宽扩展和近端听力增强技术等语音增强算法将被应用并适应于噪声环境下低容量便携式设备的实时工作。这项研究计划的成果将对加拿大和世界其他地区的普通民众、科学界和工业部门产生影响。它们将使数百万患有听力障碍的加拿大人受益，并减少使用堵塞口腔的ppe的医务人员的工作压力。

In an aging population that is increasingly affected by hearing impairment where intelligent devices such as home assistants and wearable in-ear devices rely on speech for human-human as well as human-machine communication, changes in speech production must be assessed and well understood to optimize the performance of such devices for all talkers. Current speech technologies, including speech recognition systems, factor in disturbances on speech production such as noise, or reverberant environments, but they fail to take into account other common disturbances.  In fact, current speech technologies are trained on open--ear normal-hearing talkers and do not include the increasing population of hearing impaired talkers. Furthermore, occluding the earcanal with an intra--aural device, such as a hearing protection device or an earphone, causes changes to speech production that are detrimental to speech intelligibility, especially in noise. In addition, obstruction of the mouth with a face mask or respirator degrades speech intelligibility and the performance of these tools, which is especially relevant in the current pandemic. The long-term goal of this research program is to make speech technologies more inclusive by developing tools that are suitable and adaptable to hearing impairments, including age-induced and noise-induced hearing loss, and speech degredations, such as those caused by the use of face masks and respirators. This will be achieved by modeling changes in speech production caused by a number of disturbances, and subsequently developing speech enhancement algorithms that amend the degredations on speech intelligibility caused by these changes. Two large databases of speech sentences produced in atypical conditions will be collected and made available through open-access. To understand the changes in speech production caused by each disturbance, linear mixed models will be created for acoustic features that are tied to intelligibility such as intensity, spectral tilt, vowel duration, and formant bandwidth.  Subsequently, speech enhancement algorithms such as spectral envelope mapping, bandwidth extension, and near-end listening enhancement techniques will be employed and adapted to work in real time on low capacity portable devices in noisy environments. The outcomes of this research program will have impact on the general population, the scientific community, and the industrial sector, in Canada and other parts of the world. They will benefit millions of Canadians suffering from hearing impairment and reduce work related stress for medical staff that use PPEs that obstruct the mouth.