Towards more inclusive speech technologies: speech enhancement in atypical conditions

在日益受到听力障碍影响的老龄化人口中，智能设备（如家庭助理和可穿戴入耳式设备）依赖语音进行人与人以及人机通信，必须评估并充分理解语音产生的变化，以优化此类设备对所有说话者的性能。当前的语音技术，包括语音识别系统，考虑了对语音产生的干扰，例如噪声或混响环境，但它们没有考虑其他常见的干扰。 事实上，目前的语音技术是在听力正常的开放式谈话者上训练的，并且不包括日益增加的听力受损谈话者。此外，用耳内装置（例如听力保护装置或耳机）阻塞耳道会导致语音产生的变化，这对语音清晰度（尤其是噪声）有害。此外，用面罩或呼吸器阻塞口腔会降低语言清晰度和这些工具的性能，这在当前的大流行中尤其相关。 该研究计划的长期目标是通过开发适合和适应听力障碍的工具，使语音技术更具包容性，包括年龄引起的和噪音引起的听力损失，以及语音退化，如使用面罩和呼吸器引起的语音退化。这将通过模拟由一些干扰引起的语音产生的变化来实现，随后开发语音增强算法，该算法修正由这些变化引起的语音清晰度的程度。将收集非典型条件下产生的言语句子的两个大型数据库，并通过开放存取提供。为了了解每种干扰引起的语音产生的变化，将为与可懂度相关的声学特征（如强度、频谱倾斜、元音持续时间和共振峰带宽）创建线性混合模型。 随后，语音增强算法，如频谱包络映射，带宽扩展，和近端收听增强技术将被采用和适应工作在真实的时间在低容量的便携式设备在嘈杂的环境。这项研究计划的成果将对加拿大和世界其他地区的普通民众、科学界和工业部门产生影响。他们将使数百万患有听力障碍的加拿大人受益，并减轻使用PPE阻塞口腔的医务人员的工作压力。

In an aging population that is increasingly affected by hearing impairment where intelligent devices such as home assistants and wearable in-ear devices rely on speech for human-human as well as human-machine communication, changes in speech production must be assessed and well understood to optimize the performance of such devices for all talkers. Current speech technologies, including speech recognition systems, factor in disturbances on speech production such as noise, or reverberant environments, but they fail to take into account other common disturbances.  In fact, current speech technologies are trained on open--ear normal-hearing talkers and do not include the increasing population of hearing impaired talkers. Furthermore, occluding the earcanal with an intra--aural device, such as a hearing protection device or an earphone, causes changes to speech production that are detrimental to speech intelligibility, especially in noise. In addition, obstruction of the mouth with a face mask or respirator degrades speech intelligibility and the performance of these tools, which is especially relevant in the current pandemic. The long-term goal of this research program is to make speech technologies more inclusive by developing tools that are suitable and adaptable to hearing impairments, including age-induced and noise-induced hearing loss, and speech degredations, such as those caused by the use of face masks and respirators. This will be achieved by modeling changes in speech production caused by a number of disturbances, and subsequently developing speech enhancement algorithms that amend the degredations on speech intelligibility caused by these changes. Two large databases of speech sentences produced in atypical conditions will be collected and made available through open-access. To understand the changes in speech production caused by each disturbance, linear mixed models will be created for acoustic features that are tied to intelligibility such as intensity, spectral tilt, vowel duration, and formant bandwidth.  Subsequently, speech enhancement algorithms such as spectral envelope mapping, bandwidth extension, and near-end listening enhancement techniques will be employed and adapted to work in real time on low capacity portable devices in noisy environments. The outcomes of this research program will have impact on the general population, the scientific community, and the industrial sector, in Canada and other parts of the world. They will benefit millions of Canadians suffering from hearing impairment and reduce work related stress for medical staff that use PPEs that obstruct the mouth.