权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

An Implementation of Multimedia Communication in Knowledge Acquisition Process

多媒体通信在知识获取过程中的实现

基本信息

批准号：
07680409
负责人：
ENDO Tsutomu
金额：
$ 1.41万
依托单位：
Oita University
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (C)
财政年份：
1995
资助国家：
日本
起止时间：
1995 至 1997
项目状态：
已结题

项目摘要

This research intends to implement a multimedia communication system for problem solving and knowledge acquisition, focusing on the first grade mathematics.1.Dialogue-based problem solving system.We have developed a problem solving system based on co-reference between drill texts of the first grade mathematics and dialogue with a tescher. It has been designed with emphasis on the issue in what situation of problem solving process it should utter, what sentence it should utter, and how it should interpret the response using contextual information. The constituents of the context are surface and case strusture of utterances, intention and sttention of the speaker, situation of dialogue, and world knowledge.2.Cooperative understanding of utterances and gestures.The teacher in general utters a sentence while he/she makes such a gesture as pointing to some objects in the text, or drawing some pictures. We have proposed a method of cooperative understanding of utterances and gestures in dialogue. The point of interest in the gesture is the movement of teacher's fingertip or pen. The location of the tip of the pen is detected from moving imades. Then the relationship between the features extracted from the moving points and linguistic information in the utterance as well as interpreting procedure of the gesture were presented.3.Knowledge acquisition, speech processing and face recognition.First, we made the assumption that knowledge acquired from dialogue is heuristics for problem solving, and presented a method of knowledge generalization and reuse. Then we confirmed that the speech interface can be used for dialogue-based problem solving process by experiment of speech recognition. Next, facial expression is important information in multimodal dialogue. We have proposed a method of detecting face elements such as eyes, nose and mouth. It uses local autocorrelation features, discriminant analysis, and genetic algorithm.

本研究旨在以一年级数学为研究对象，实现一个多媒体交流的问题解决与知识获取系统：1.基于对话的问题解决系统：我们开发了一个基于一年级数学习题集和与教师对话的问题解决系统。它的设计重点是在什么情况下的问题解决过程中，它应该说，什么句子，它应该说，以及它应该如何解释的响应使用上下文信息的问题。语境的构成要素包括话语的表层和格结构、说话者的意图和持续、对话情境和世界知识。2.话语和手势的合作理解。教师通常一边说一句话，一边做出手势，例如指向文本中的某些物体，或画一些图片。我们提出了一种在对话中合作理解话语和手势的方法。手势中的兴趣点是教师指尖或笔的运动。从移动图像中检测笔尖的位置。3.知识获取、语音处理和人脸识别首先假设从对话中获取的知识是解决问题的工具，提出了一种知识泛化和重用的方法。通过语音识别实验，验证了该语音界面可以用于基于对话的问题求解过程。其次，面部表情是多模态对话中的重要信息。我们提出了一种方法来检测人脸的元素，如眼睛，鼻子和嘴。它使用局部自相关特征，判别分析和遗传算法。