Towards more inclusive speech technologies: speech enhancement in atypical conditions

迈向更具包容性的语音技术：非典型条件下的语音增强

• 批准号: RGPIN-2021-03182
• 负责人: Bouserhal, Rachel
• 金额: $ 2.04万
• 依托单位: École de technologie supérieure
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=754412
• 关键词: Towards; more; inclusive; speech; technologies

In an aging population that is increasingly affected by hearing impairment where intelligent devices such as home assistants and wearable in-ear devices rely on speech for human-human as well as human-machine communication, changes in speech production must be assessed and well understood to optimize the performance of such devices for all talkers. Current speech technologies, including speech recognition systems, factor in disturbances on speech production such as noise, or reverberant environments, but they fail to take into account other common disturbances.  In fact, current speech technologies are trained on open--ear normal-hearing talkers and do not include the increasing population of hearing impaired talkers. Furthermore, occluding the earcanal with an intra--aural device, such as a hearing protection device or an earphone, causes changes to speech production that are detrimental to speech intelligibility, especially in noise. In addition, obstruction of the mouth with a face mask or respirator degrades speech intelligibility and the performance of these tools, which is especially relevant in the current pandemic. The long-term goal of this research program is to make speech technologies more inclusive by developing tools that are suitable and adaptable to hearing impairments, including age-induced and noise-induced hearing loss, and speech degredations, such as those caused by the use of face masks and respirators. This will be achieved by modeling changes in speech production caused by a number of disturbances, and subsequently developing speech enhancement algorithms that amend the degredations on speech intelligibility caused by these changes. Two large databases of speech sentences produced in atypical conditions will be collected and made available through open-access. To understand the changes in speech production caused by each disturbance, linear mixed models will be created for acoustic features that are tied to intelligibility such as intensity, spectral tilt, vowel duration, and formant bandwidth.  Subsequently, speech enhancement algorithms such as spectral envelope mapping, bandwidth extension, and near-end listening enhancement techniques will be employed and adapted to work in real time on low capacity portable devices in noisy environments. The outcomes of this research program will have impact on the general population, the scientific community, and the industrial sector, in Canada and other parts of the world. They will benefit millions of Canadians suffering from hearing impairment and reduce work related stress for medical staff that use PPEs that obstruct the mouth.

在日益受到听力障碍影响的老龄化人口中，家庭助理和可穿戴式入耳设备等智能设备依赖语音进行人机交流，因此必须评估和充分了解语音产生的变化，以优化所有说话者的此类设备的性能。当前的语音技术，包括语音识别系统，考虑了噪声或混响环境等对语音产生的干扰，但它们没有考虑到其他常见的干扰。事实上，当前的语音技术是针对听力正常的开耳人进行培训的，并不包括越来越多的听力受损的人。此外，用听力保护装置或耳机等耳内设备堵塞耳道会导致语音产生的变化，这对语音清晰度是有害的，尤其是在噪音中。此外，使用口罩或呼吸器阻塞口腔会降低语言清晰度和这些工具的性能，这在当前的大流行中尤为重要。这项研究计划的长期目标是通过开发适合和适应听力损伤的工具，使语音技术更具包容性，包括年龄和噪音导致的听力损失，以及语音程度，如使用口罩和呼吸器造成的程度。这将通过对一些干扰导致的语音产生变化进行建模，并随后开发语音增强算法来修正这些变化对语音清晰度的影响，从而实现这一目标。将收集在非典型条件下产生的两个大型演讲句子数据库，并通过开放获取提供。为了了解每个干扰引起的语音产生的变化，将为与清晰度相关的声学特征创建线性混合模型，如强度、频谱倾斜、元音持续时间和共振峰带宽。随后，将采用诸如频谱包络映射、带宽扩展和近端收听增强技术等语音增强算法，并对其进行调整，以在噪声环境中的低容量便携式设备上实时工作。这项研究计划的结果将对加拿大和世界其他地区的普通民众、科学界和工业部门产生影响。它们将使数百万患有听力障碍的加拿大人受益，并减轻使用PPE阻塞口腔的医务人员的工作压力。