Study of Multi-Modal Interface based on Simultaneous Understanding of Spoken Language

点击翻译按钮获取中文摘要

The spoken dialog systems has been studied to help persons who cannot use hands or pointing devices and who do not familiar with keyboard nor computers. In this research project, we develop the system named Sync/Mail, which is a multi-modal drawing tool which incrementally understands speech and quickly responds for inputs. To implement the system, we investigate following topics.(1) Incremental Interpretation of Spoken LanguageConventional natural language processing systems manage natural language input by sentence-by-sentence basis process. Our incremental system processes the input by word-by-word basis. This method enables the spontaneous reply of input.(2) Incremental Continuous Speech Recognition(3) Method of Integration for Multi-Domain SystemsTo build the unified system for multi-domain task, it is required the method to integrate the several single-domain dialogue system into the one unified system. We proposed an extensible and scalable architecture for the multi-domain, multi-modal conversational systems.