An Implementation of Multimedia Communication in Knowledge Acquisition Process

多媒体通信在知识获取过程中的实现

基本信息

批准号：
07680409
负责人：
ENDO Tsutomu
金额：
$ 1.41万
依托单位：
Oita University
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (C)
财政年份：
1995
资助国家：
日本
起止时间：
1995 至 1997
项目状态：
已结题

项目摘要

This research intends to implement a multimedia communication system for problem solving and knowledge acquisition, focusing on the first grade mathematics.1.Dialogue-based problem solving system.We have developed a problem solving system based on co-reference between drill texts of the first grade mathematics and dialogue with a tescher. It has been designed with emphasis on the issue in what situation of problem solving process it should utter, what sentence it should utter, and how it should interpret the response using contextual information. The constituents of the context are surface and case strusture of utterances, intention and sttention of the speaker, situation of dialogue, and world knowledge.2.Cooperative understanding of utterances and gestures.The teacher in general utters a sentence while he/she makes such a gesture as pointing to some objects in the text, or drawing some pictures. We have proposed a method of cooperative understanding of utterances and gestures in dialogue. The point of interest in the gesture is the movement of teacher's fingertip or pen. The location of the tip of the pen is detected from moving imades. Then the relationship between the features extracted from the moving points and linguistic information in the utterance as well as interpreting procedure of the gesture were presented.3.Knowledge acquisition, speech processing and face recognition.First, we made the assumption that knowledge acquired from dialogue is heuristics for problem solving, and presented a method of knowledge generalization and reuse. Then we confirmed that the speech interface can be used for dialogue-based problem solving process by experiment of speech recognition. Next, facial expression is important information in multimodal dialogue. We have proposed a method of detecting face elements such as eyes, nose and mouth. It uses local autocorrelation features, discriminant analysis, and genetic algorithm.

这项研究旨在实施一个多媒体通信系统，以解决问题和知识获取，重点关注一年级数学。1。基于Dialogue的问题解决系统。我们已经基于一年级数学的练习文本与Tescher对话之间的共同参考，开发了一个问题解决系统。它的设计重点是在解决问题的过程中应该说出的问题，应该说出什么句子以及如何使用上下文信息来解释响应的问题。上下文的组成部分是对讲话者的话语，意图和口语的表面和案例强烈，对话的状况以及世界知识。2。对话语和手势的合作理解。一般而言，老师在句子中提出了一个句子，而他/她将其表现为指向文本中的某些对象或绘制某些图片的手势。我们提出了一种合作理解对话中话语和手势的方法。手势的兴趣是教师指尖或笔的运动。从移动的iMades中检测到笔尖的位置。然后，提出了从动态点提取的特征与语言信息以及手势的解释过程之间的关系。3。知识获取，语音处理和面部识别。首先，我们假设从对话中获取的知识是解决问题的启发式知识，并提出了知识通用和重复使用的方法。然后，我们确认语音界面可通过通过语音识别实验来用于基于对话的问题解决过程。接下来，面部表达是多模式对话中的重要信息。我们提出了一种检测眼睛，鼻子和嘴巴等面部元素的方法。它使用局部自相关特征，判别分析和遗传算法。