Development of a multi-modal dialogue system and a tool for a spoken dialogue system

在本研究中，我们发布了一个连续语音识别的免费软件，它由客户机/服务器结构组成，因此用户可以有效地使用该软件作为语音输入方式，在标准PC上开发一个口语演示系统或多模态对话系统。为了实现自然的人机交互，我们开发了一个多模态的观光引导系统，该系统具有1）语音输入/输出，2）触摸屏输入（在地图上/菜单中）和3）图形/文本输出（地图、照片、菜单和历史记录）。在此基础上，我们实现了一个带有真实的人脸图像/动画和录音/合成语音的智能体界面，并通过任务完成和问卷调查对界面和整个系统进行了评价。评价实验表明了该方法的有效性。近年来，对语音识别和语言处理的鲁棒性和可用性的研究已经建立，语音识别系统和对话系统已经开发出实际应用。但是，如果这些系统将成为实用的，这是重要的，不仅是这些基本技术，而且还应开发的可移植性和可扩展性的技术。富士观光，指导到三川观光指导。设计了一个与领域无关的口语对话系统数据库检索平台，并将该平台应用于文献检索系统中。

In this research, we published a continuous speech recognition free software which consists of a clients server architecture and thus a user can effectively use this software as a means of speech input modality for developing a spoken dealogue system or multimodal dialogue system on standard PCs.In order to realize natural human-macbin interaction, we have developed a multi-modal sightseeing guidance system with 1) speech input / output, 2) touch screen input (on map/in menu) and 3) graphical/text output (map, photograph, menu and dealogue historiy). Furthermore, we implemented an agent interface wiht real face image / animation and recorded speech / synthesized speech to the system, and carried out evaluation experiments which consist of task completions and questionnaires to evaluate the interface and whole system. The evaluation experiments showed the effectiveness.Recently the study of robustenss and usability for speech recognition and language processing has been established, and speech recognition systems and dialogue systems have been developed to be practical use. But if these systems will become practical, it is important that not only those fundamental techniques but also the techniques of portability and expansibility should be developed.Based on this consideration, we examined our system in portability by transfering the domain of the system form the Mt. Fuji sightseeing, guidance to the Mikawa sightseeing guidance. Also we designed a domain independent platform of spoken dialogue system for database retrival, and applied the platform to a literature retrieval system.