Auditory-motor control analysis of voice production in hearing impaired speakers by means of Machine Learning

言语产生是一个高度复杂的过程，涉及呼吸、喉和口腔运动系统的协调，以及涉及运动、体感和听觉任务的大型脑区网络。听觉反馈在言语运动控制（SMC）系统的调节中起着重要的作用。众所周知，听觉剥夺，由于听力损失，可能会导致显着恶化的发音过程。然而，很少有人知道有关的神经反馈网络和潜在的机制还没有完全理解。本研究的中心目标是在听力障碍患者的言语运动控制机制是确定的影响，干扰听觉输入的听觉动觉过程。通过应用和分析基于多传感器的数据，包括喉部高速成像，脑电图（EEG）和声学语音信号，该项目旨在描绘感知和运动之间的相互作用。音高-移位-反射（PSR）被称为对修正听觉反馈的音高变化的反应，将被用作研究SMC相关过程的范例。将记录听力受损患者和听力正常受试者的数据。将使用机器学习技术对基于多传感器的数据进行分析，以识别反映SMC过程的生理决定性特征。为了区分群体，将研究参数驱动和数据驱动的机器学习方法。这产生了代表SMC恶化的临床相关参数。本研究的主要目标是将这些参数与患者的特定听力学特征（如听力损失的程度、持续时间和频率范围、患者年龄、频差阈和言语感知）相关联。通过比较PSR的变化与这些听力学特征更深入的了解SMC恶化的类型和程度，由于听觉下降。该项目的创新科学方面是（1）使用高速视频内窥镜，允许在PSR期间直接观察喉部动力学。(2)机器学习方法将被应用于揭示正常听力受试者和听力受损者之间高速视频、EEG和声学数据中潜在SMC参数的差异。(3)确定的SMC参数和患者特定听力学特征的多元回归分析。

Speech production is a highly complex process involving the coordination of the respiratory, laryngeal, and oral motor systems as well as a large network of brain regions being involved in motor, somatosensory, and auditory tasks. The auditory feedback plays an important role in tuning the speech motor control (SMC) system. It is well known that auditory deprivation, due to hearing loss, may result in significant deteriorations of articulatory processes. However, little is known about the involved neuro-feedback networks and the underlying mechanisms are not fully understood yet.The central objective in this study on Speech Motor Control mechanisms in hearing impaired patients is the identification of the impact of disturbed auditory input on audio-kinesthetic processes. By applying and analyzing multi-sensor based data including laryngeal high-speed imaging, electroencephalography (EEG), and the acoustic voice signal the project aims to delineate the interaction between perception and motoric. The Pitch-Shift-Reflex (PSR) known as pitch changes in response to modified auditory feedback will be used as a paradigm for investigating SMC related processes.Data for both hearing impaired patients and normal hearing subjects will be recorded. Analysis of the multi-sensor based data will be performed using machine learning techniques to identify physiological conclusive features that reflect SMC processes. For differentiating between groups, parameter-driven and data-driven machine learning approaches will be investigated. This yields clinical relevant parameters which represent SMC deterioration. The major goal of the study will be pursued by correlating these parameters with patient specific audiological characteristics as degree, duration and frequency range of hearing loss, age of patients, frequency difference limen, and speech perception. By comparing PSR changes with these audiological characteristics deeper insight in kind and extent of SMC deterioration due to auditory decline is expected. Innovative scientific aspects of the project are (1) the use of high-speed-video endoscopy allowing for direct observation of the laryngeal dynamics during the PSR. (2) Machine learning approaches will be applied to reveal differences in underlying SMC parameters in high-speed videos, EEG, and acoustic data between normal hearing subjects and hearing impaired. (3) Multi-regression analysis for identified SMC parameters and patient specific audiological characteristics.