Construction of speech acquisition mechanism based on sensory information

我们在语音发音过程中感觉反馈的扰动研究中研究了基于感觉信息的语音习得机制，并利用机械说话机器人模仿人类的语音产生进行了建构性研究。微扰研究：我们使用微扰范式研究了感觉反馈对言语产生的影响。在掩蔽的听觉和触觉反馈条件下，外部扰动意外地应用于语音发声过程中的腭部形状。然后，我们用fMRI观察了舌头对扰动的代偿反应过程中的大脑活动。我们发现这种代偿反应是由中枢神经系统的神经适应机制带来的。语音机器人研究：我们开发了一种模仿人类语音产生过程的三维语音机器人(机械语音合成器)。该机器人以人工机械肺、声门、舌头、颌骨、嘴唇和舌膜为语音器官…更多由伺服机构动态控制。语音是由自振声带产生的声门源、气流通过声道狭窄处产生的随机源和时变的声道声学共振产生的。通过前馈-反馈机构对说话机器人进行动态控制。自动训练控制规划以获得所需的语音、声学和感觉信息，如舌头和上颚之间的触觉信息和口腔压力。提出了一种基于隐马尔可夫模型的随机语音倒置方法，将语音声学信息映射到发音位置。此外，我们还提出了一种基于雅各布更新规则的规划方法，用于根据感觉信息确定关节位置。还研究了基于正反向神经网络模型的喉部控制规划，以确定机器人喉部的参数，以获得所需的音调频率、清音信息和语音质量。较少

We have been studied speech acquisition mechanism based on sensory information in perturbation studies of sensory feedback during speech articulation and constructive studies by using mechanical talking robot mimicking human speech production.1. Perturbation Study : We investigated sensory feedback effects on speech production by using a perturbation paradigm. The external perturbation was unexpectedly applied on the palatal shape during speech utterance under the masked auditory and tactile feedback conditions. Then, we observed brain activity by using fMRI during the compensatory response by the tongue to the perturbation. We found that the compensatory response was brought by the neural adaptation mechanism in the central nerves system.2. Talking Robot Study : We have developed an 3 dimensional talking robot (mechanical speech synthesizer) which mimicking human speech production process. This robot has artificial mechanical lung, glottis, tongue, jaw, lips and velum as speech organs … More which are dynamically controlled by servo mechanism. The speech sounds are produced by glottal source generated by self-oscillating vocal cords, random source generated by passing the air flow through the narrow constriction in the vocal tract, and time varying vocal tract acoustic resonance. The talking robot is dynamically controlled by feed-forward-feedback mechanism. The control planning is automatically trained for a desired speech acoustics and sensory information like tactile information between the tongue and the palate and oral pressure. We proposed a stochastic speech inversion method for mapping the speech acoustics to the articulatory positions based on Hidden Markov Model. Also, we proposed a planning method based on a Jacobean updating rule to determine the articulatory positions from the sensory information. The larynx control planning is also studied based on forward-inverse neural network model for determining the robot parameters of larynx for a desired acoustics like pitch frequency, voice-unvoiced information, and voice quality. Less