Recognition of Presentation by Integration of Visual and Linguistic Information

通过整合视觉和语言信息来识别演示

• 批准号: 06452396
• 负责人: OHTA Yuichi
• 金额: $ 3.52万
• 依托单位: University of Tsukuba
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for General Scientific Research (B)
• 财政年份: 1994
• 资助国家: 日本
• 起止时间: 1994 至 1995
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-06452396/
• 关键词: Video Indexing; Gesture Dialog Relation; Human Behavior Understanding; Image Understanding; Media Integration; Natural Language Processing; Human Interface; Presentation; 画像認識; プレゼンテーションの理解; 動画像処理; ジェスチャー認識; 自然言語とパターン情報の統合

In the understanding of human behaviors, there is a lot of ambiguities which depend on the situation. It is because few strict laws or rules are applicable throughout a wide variety of situations. We have examined the relationship between the gestures and the context of situation which typically represented in the spoken dialog. We proposed a novel framework to understand the behaviro in oral presentations.1. Human behavior understanding in oral presentation : We developed a method to extract visual keys from presentation images. We also developed a method to extract linguistic keys from spoken words in the presentation. A novel framework was developed to integrate the both keys to resolve the intention of the presenter.2. Temporal structure analysis of video by image and sound processing : We have developed a method to estimate the temporal structure of a video sequence considering the contents and the intention of the author. it uses the visual and sound keys. Television commercials are used as the target presentation images.3. Knowledge extraction from diagram and text : We developed a new framework for knowledge extraction from written texts and diagrams and utilization of the obtained knowledge for the automatic organization of flexible hyper-media.

在对人类行为的理解中，存在着许多取决于情境的模糊性。这是因为很少有严格的法律或规则适用于各种各样的情况。我们已经研究了手势和情景之间的关系，这通常表现在口语对话。我们提出了一个新的框架来理解口头陈述中的重复。口头演示中的人类行为理解：我们开发了一种从演示图像中提取视觉键的方法。我们还开发了一种方法，从演讲中的口语单词中提取语言键。提出了一个新的框架，将两个关键词结合起来，以解决演示者的意图.通过图像和声音处理对视频进行时间结构分析：我们开发了一种方法来估计视频序列的时间结构，同时考虑作者的内容和意图。它使用视觉和声音键。电视广告作为目标呈现图像.从图表和文本中提取知识：我们开发了一个新的框架，用于从书面文本和图表中提取知识，并利用所获得的知识自动组织灵活的超媒体。

项目成果

中村裕一: "Knowledge Extraction from Diagram and Text for Media Integration" Proc. Int. Conference on Multimedia Computing and Systems. (to be published). (1995)

Yuichi Nakamura：“从图表和文本中提取知识以实现媒体集成”，多媒体计算和系统国际会议（即将出版）。

Yuichi Nakamura, Masashi Nishitani, Yuichi Ohta: "Human Behavior Understanding in Oral Presentation" IEICE Technical Report SIG-PRU. Vol.95-143. 51-56 (1995)

Yuichi Nakamura、Masashi Nishitani、Yuichi Ohta：“口头演示中的人类行为理解”IEICE 技术报告 SIG-PRU。

角 保志: "Detection of Face Orientation and Facial Components Using Distributed Appearance Model" Int. Workshop on Automatic Face-and Gesture Recognition. 254-259 (1995)

Yasushi Kado：“使用分布式外观模型检测面部方向和面部成分”自动面部和手势识别研讨会 254-259 (1995)

Yukiyo Uehori, Mitsuhiro Murata, Yuichi Nakamura, Yuichi Ohta: "Temporal Structure Analysis of Television Commercial by Image and Sound Processing" IEICE Technical Report SIG-PRU. Vol.95-159. 9-12 (1995)

Yukiyo Uehori、Mitsuhiro Murata、Yuichi Nakamura、Yuichi Ohta：“通过图像和声音处理进行电视广告的时间结构分析”IEICE 技术报告 SIG-PRU。

中村 裕一: "Knowledge Extraction from Diagram and Text for Media Integration" Proc. Int. Conference on Multimedia Computing and Systems. (to be published). (1996)

Yuichi Nakamura：“从图表和文本中提取知识以实现媒体集成”，多媒体计算和系统国际会议（即将出版）。