Design and implementation of spiking neural network based speech enhancement algorithms

数以百万计的加拿大人每天使用移动的或可穿戴多媒体设备，如平板电脑、智能手机、智能手表或智能眼镜。所有这些设备都包括不同的语音处理模块，例如语音编码器或自动语音识别系统，例如Apple的Siri或Amazon的Alexa。由于多媒体设备的移动性越来越高，这些模块比以往任何时候都更容易受到各种噪声环境的影响，但它们的性能受到噪声的严重影响。为了限制这些模块在噪声存在下的性能下降，使用语音增强（SE）算法来降低噪声而不影响语音质量。尽管经过数十年的研究，这些SE算法的性能在很大程度上是次优的，特别是当只使用一个麦克风时。与此形成鲜明对比的是，听觉系统能够很好地处理噪音。事实上，在相对嘈杂的环境中，人类很容易听懂对话。对听觉系统中的神经处理进行建模的一种方法是使用尖峰神经网络（SNN）。SNN比经典神经网络更接近生物神经元。此外，SNN已经被发现比经典方法在计算上更强大，并且具有比CPU快得多的硬件实现，同时消耗更少的能量。 在这项研究中，我建议设计计算效率高的SNN为基础的SE算法，将模拟的听觉系统进行的处理。为此，我将首先开发SNN，它将对听觉系统中的噪声处理进行建模。然后，我将设计基于单通道SNN的SE算法。最后，这些基于SNN的SE算法将在低功耗神经形态芯片上实现。拟议的研究将有助于几个研究领域，如神经科学，机器学习和语音处理。更具体地说，在噪声外部刺激存在下神经活动的新SNN模型的开发将有助于提高我们对噪声存在下大脑神经处理的理解。此外，SNN将允许开发新的SE估计器，该估计器将集成这些模型，并将进一步在神经形态芯片中实现。在该研究计划下开发的SE算法最终将被集成到商业应用中，它们将提高使用它们的众多设备的性能，例如平板电脑，智能手机，眼镜或手表。加拿大一直是语音技术的重要参与者，因此这项研究计划将有助于保持加拿大这一重要产业的高质量。

Mobile or wearable multimedia devices such as tablets, smartphones, smartwatches or smart glasses are used daily by millions of Canadians. All of these devices include different speech processing modules such as speech coders or automatic speech recognition systems, e.g. Apple's Siri or Amazon's Alexa. Due to the increasing mobility of multimedia devices, these modules are more than ever subjected to diverse noisy environments but, yet, their performance is seriously affected by the presence of noise. In order to limit the decrease in performance of these modules in the presence of noise, speech enhancement (SE) algorithms are used to reduce the noise without affecting the speech quality. Despite decades of research, the performance of these SE algorithms are largely sub-optimal, especially when using only one microphone. In sharp contrast, the auditory system deals very well with noise. In fact, it is somewhat easy for humans to follow a conversation in a relatively noisy environment. One approach to model neural processing in the auditory system is to use spiking neural networks (SNN). SNN mimic biological neurons more closely than classic neural networks. Moreover, SNN have been found to be more computationally powerful than classic approaches and have hardware implementations that are much faster than a CPU while consuming much less energy. In this research, I propose to design computationally efficient SNN-based SE algorithms that will model the processing performed by the auditory system. To do so, I will first develop SNN that will model the processing of noise in the auditory system. I will then design single-channel SNN-based SE algorithms. Finally, these SNN-based SE algorithms will be implemented on low-consumption neuromorphic chips. The proposed research will contribute to several fields of research such as neuroscience, machine learning and speech processing. More specifically, the development of new SNN models of neural activity in the presence of a noisy external stimulus will contribute to improving our understanding of the neural processing in the brain in the presence of noise. Moreover, the SNN will allow the development of novel SE estimators that will integrate such models and that will be further implemented in neuromorphic chips. The SE algorithms developed under this research program will eventually be integrated in commercial applications and they will improve the performance of the numerous devices that use them, e.g. tablets, smart phones, glasses or watches. Canada has always been an important player in speech technology and this research program will thus contribute to maintaining the high quality of this important Canadian industry.