Recognition of Presentation by Integration of Visual and Linguistic Information

点击翻译按钮获取中文摘要

In the understanding of human behaviors, there is a lot of ambiguities which depend on the situation. It is because few strict laws or rules are applicable throughout a wide variety of situations. We have examined the relationship between the gestures and the context of situation which typically represented in the spoken dialog. We proposed a novel framework to understand the behaviro in oral presentations.1. Human behavior understanding in oral presentation : We developed a method to extract visual keys from presentation images. We also developed a method to extract linguistic keys from spoken words in the presentation. A novel framework was developed to integrate the both keys to resolve the intention of the presenter.2. Temporal structure analysis of video by image and sound processing : We have developed a method to estimate the temporal structure of a video sequence considering the contents and the intention of the author. it uses the visual and sound keys. Television commercials are used as the target presentation images.3. Knowledge extraction from diagram and text : We developed a new framework for knowledge extraction from written texts and diagrams and utilization of the obtained knowledge for the automatic organization of flexible hyper-media.