Studies on conversation systems with understanding and generating functions of linguistic and para-linguistic information

作为一种研究自然口语交际基本要素的工具，开发了一个具有语言和副语言信息理解和生成功能的口语对话系统原型，尽管在语音识别和合成方面已经进行了许多优秀的研究，但还没有一个实用的口语对话系统让我们满意。其中一个原因是，大多数口语对话系统不处理副语言信息。对副语言信息的定量理解不足以构成自然的对话系统。在本研究中，我们尝试实现了多种构件化技术和一个对话机器人平台，作为揭示副语言量化卷的工具。1)安装在机器人头部的四线定向传声器的声音定位和分离方法；2)基于波形合成和高质量语音转换的高质量语音合成方法，用于表达副语言信息；3)基于韵律信息作为语音信息中的副语言信息的姿态识别和反向通道反馈生成方法；4)作为视觉信息中的副语言信息的头部手势识别和面部表情识别方法；5)作为口语对话系统平台开发的仿人机器人ROBISUKE；6)面向消息的机器人体系结构，MONEA，对上述模块的集成提出了建议。未来的工作包括定量寻找自然对话的必要要求的实验。

As a tool for investigating fundamental elements of natural spoken language communication, a prototype of spoken dialogue system with understanding and generating functions of linguistic and para-linguistic information was developed.Although many excellent studies on speech recognition and synthesis have been conducted, there exists no practical spoken dialogue system which satisfies us. One of the reasons is that most spoken dialogue systems did not deal with para-linguistic information. The quantitative understanding for para-linguistic information is not sufficient enough to make natural conversation system. In this study, we tried realizing many component technologies and a platform of conversation robot as tools to reveal the quantitative rolls of para-language.In particular, the following outcomes were obtained. 1) the sound localization and separation methods using the four-line directivity microphone mounted on head of robot, 2) the high quality speech synthesis method based on the waveform synthesis and the high quality voice conversion method for expressing para-linguistic information, 3) the method of attitude recognition and back-channel feedback generation based on the prosodic information as para-linguistic information in speech information, 4) the method of head gesture recognition and facial expression recognition as para-linguistic information in visual information, 5) humanoid robot "ROBISUKE" developed as the platform of the spoken dialogue system, and 6) Message Oriented RObot Architecture, MONEA, proposed for the integration of the abovementioned modules.Future work includes the experiment for finding out the necessary requirement for natural conversation quantitatively.