A Study on a framework of spontaneous communication depending on dialogue situation

在这项研究中，我们研究了一个与用户自发交互的通信系统框架，以应对实际的对话环境。传统的口语对话系统旨在有效地完成特定的语音对话任务，而自发交流系统的框架是为了将这些口语对话系统进化为自发地开始和继续口语对话的系统而开发的。为此，我们进行了三个方面的研究：(1)对对话环境的理解，旨在利用图像和语音信号进行高级人类识别和口语对话识别；(2)自发交流管理模型，为口语对话的开始、继续和结束建模；(3)语音生成和动作表达技术，即如何通过话语或动作来呈现系统的意图。在研究(i)中，开发了基于立体摄像机的人体姿态估计方法。同时利用图像的空间深度、人体或衣服的形状和纹理信息，实现了对人体姿态的准确估计。此外，还研究了话语意图的估计。利用句尾和单词n -gram的特征，实现了更准确的话语意图。在研究（ii）中，开发了机器人与人开始交流、继续交流和结束交流的模型。为此，我们开发了对话伙伴的心理状态。在研究中，开发了一种笑声语音生成技术。在对人类笑声进行声学分析的基础上，实现了笑声和笑语的合成。通过这三项研究，建立了自发交际的基本框架。

In this study, we examined a framework of communication systems that interact with users spontaneously so as to cope with practical dialogue environments. While conventional spoken dialogue systems aimed to efficiently achieve specific speech-dialogue tasks, the framework of spontaneous communication system was developed in order to evolve these spoken dialogue systems to one that spontaneously start and continue spoken dialogues.For this purpose, three studies were conducted : (i) understanding of dialogue environment, which aims advanced human recognition and spoken dialogue recognition using image and speech signal, (ii) spontaneous communication management model, which models how to start, continue and end spoken dialogues, and(iii) speech generation and motion expression technology, which is how to present intentions of the system by utterances or motions.In the study(i), human pose estimation using stereo camera was developed. Simultaneous adoption of information of space depth of images, and shapes and textures of either human bodies or clothes realized accurate estimation of human poses. In addition, estimation of utterance intention was studied. Using characteristics of end of sentences and word N-grams, more accurate utterance intention was achieved.In the study(ii), models for a robot to start communication with a human, to continue it, and to end it were developed. we developed mental-state of a conversational partner for these purposes.In the study a speech generation technique for laughter was developed. Based on acoustical analyses of human laughter syntheses of both laughter and laughter-speech were realized. Throughout these three studies, basic framework of spontaneous communication was established.