Towards more inclusive speech technologies: speech enhancement in atypical conditions

点击翻译按钮获取中文摘要

In an aging population that is increasingly affected by hearing impairment where intelligent devices such as home assistants and wearable in-ear devices rely on speech for human-human as well as human-machine communication, changes in speech production must be assessed and well understood to optimize the performance of such devices for all talkers. Current speech technologies, including speech recognition systems, factor in disturbances on speech production such as noise, or reverberant environments, but they fail to take into account other common disturbances.  In fact, current speech technologies are trained on open--ear normal-hearing talkers and do not include the increasing population of hearing impaired talkers. Furthermore, occluding the earcanal with an intra--aural device, such as a hearing protection device or an earphone, causes changes to speech production that are detrimental to speech intelligibility, especially in noise. In addition, obstruction of the mouth with a face mask or respirator degrades speech intelligibility and the performance of these tools, which is especially relevant in the current pandemic. The long-term goal of this research program is to make speech technologies more inclusive by developing tools that are suitable and adaptable to hearing impairments, including age-induced and noise-induced hearing loss, and speech degredations, such as those caused by the use of face masks and respirators. This will be achieved by modeling changes in speech production caused by a number of disturbances, and subsequently developing speech enhancement algorithms that amend the degredations on speech intelligibility caused by these changes. Two large databases of speech sentences produced in atypical conditions will be collected and made available through open-access. To understand the changes in speech production caused by each disturbance, linear mixed models will be created for acoustic features that are tied to intelligibility such as intensity, spectral tilt, vowel duration, and formant bandwidth.  Subsequently, speech enhancement algorithms such as spectral envelope mapping, bandwidth extension, and near-end listening enhancement techniques will be employed and adapted to work in real time on low capacity portable devices in noisy environments. The outcomes of this research program will have impact on the general population, the scientific community, and the industrial sector, in Canada and other parts of the world. They will benefit millions of Canadians suffering from hearing impairment and reduce work related stress for medical staff that use PPEs that obstruct the mouth.