Design and implementation of spiking neural network based speech enhancement algorithms

数以百万计的加拿大人每天使用移动或可穿戴的多媒体设备，如平板电脑、智能手机、智能手表或智能眼镜。所有这些设备都包括不同的语音处理模块，如语音编码器或自动语音识别系统，如苹果的Siri或亚马逊的Alexa。由于多媒体设备的移动性越来越强，这些模块比以往任何时候都更容易受到各种噪声环境的影响，但它们的性能却受到噪声的严重影响。为了限制这些模块在存在噪声的情况下性能的下降，使用语音增强(SE)算法在不影响语音质量的情况下降低噪声。尽管经过几十年的研究，这些SE算法的性能在很大程度上是次优的，特别是在只使用一个麦克风的情况下。与之形成鲜明对比的是，听觉系统能很好地处理噪音。事实上，在相对嘈杂的环境中，人类很容易跟上对话。对听觉系统中的神经处理进行建模的一种方法是使用脉冲神经网络(SNN)。SNN比传统的神经网络更接近于模拟生物神经元。此外，SNN已经被发现在计算上比传统方法更强大，并且具有比CPU快得多的硬件实现，同时消耗的能量要少得多。在这项研究中，我建议设计计算效率高的基于SNN的SE算法，该算法将模拟听觉系统执行的处理。要做到这一点，我将首先开发SNN，它将对听觉系统中的噪声处理进行建模。然后，我将设计基于SNN的单通道SE算法。最后，这些基于SNN的SE算法将在低功耗的神经形态芯片上实现。这项拟议的研究将有助于神经科学、机器学习和语音处理等多个领域的研究。更具体地说，噪声外部刺激下神经活动的SNN模型的发展将有助于我们更好地理解噪声存在下大脑的神经处理过程。此外，SNN将允许开发新的SE估计器，该估计器将整合这些模型，并将在神经形态芯片中进一步实施。根据该研究计划开发的SE算法最终将集成到商业应用中，它们将提高使用它们的众多设备的性能，例如平板电脑、智能手机、眼镜或手表。加拿大一直是语音技术领域的重要参与者，因此，这项研究计划将有助于保持这一重要加拿大行业的高质量。

Mobile or wearable multimedia devices such as tablets, smartphones, smartwatches or smart glasses are used daily by millions of Canadians. All of these devices include different speech processing modules such as speech coders or automatic speech recognition systems, e.g. Apple's Siri or Amazon's Alexa. Due to the increasing mobility of multimedia devices, these modules are more than ever subjected to diverse noisy environments but, yet, their performance is seriously affected by the presence of noise. In order to limit the decrease in performance of these modules in the presence of noise, speech enhancement (SE) algorithms are used to reduce the noise without affecting the speech quality. Despite decades of research, the performance of these SE algorithms are largely sub-optimal, especially when using only one microphone. In sharp contrast, the auditory system deals very well with noise. In fact, it is somewhat easy for humans to follow a conversation in a relatively noisy environment. One approach to model neural processing in the auditory system is to use spiking neural networks (SNN). SNN mimic biological neurons more closely than classic neural networks. Moreover, SNN have been found to be more computationally powerful than classic approaches and have hardware implementations that are much faster than a CPU while consuming much less energy. In this research, I propose to design computationally efficient SNN-based SE algorithms that will model the processing performed by the auditory system. To do so, I will first develop SNN that will model the processing of noise in the auditory system. I will then design single-channel SNN-based SE algorithms. Finally, these SNN-based SE algorithms will be implemented on low-consumption neuromorphic chips. The proposed research will contribute to several fields of research such as neuroscience, machine learning and speech processing. More specifically, the development of new SNN models of neural activity in the presence of a noisy external stimulus will contribute to improving our understanding of the neural processing in the brain in the presence of noise. Moreover, the SNN will allow the development of novel SE estimators that will integrate such models and that will be further implemented in neuromorphic chips. The SE algorithms developed under this research program will eventually be integrated in commercial applications and they will improve the performance of the numerous devices that use them, e.g. tablets, smart phones, glasses or watches. Canada has always been an important player in speech technology and this research program will thus contribute to maintaining the high quality of this important Canadian industry.