Network Based Organization of Continuous Media by Intermodal Collaboration and Domain Knowledge Use

点击翻译按钮获取中文摘要

The goal of this research project is as follows: (1) to develop contents analysis method of continuous media such as video streams consisting of visual, auditory, textual and graphical streams, by means of intermodal collaboration that means collaborative analysis of each video stream, (2) to make the domain knowledge base to facilitate the contents analysis, and (3) to design a scheme of network based media organization called an event net. The major results of this research are classified into three issues.1. Video indexing and annotation by intermodal collaborationTo automatically produce nodes of the event net, we embodied a couple of contents analysis methods by intermodal collaboration. In particular, event based indexing for broadcasted sports video and semantical annotation in terms of players and actions were developed by considering the correspondence between multimodal streams. We verified that intermodal collaboration outperforms single stream analysis with respect to efficiency and reliability.2. Media organization based on relations between eventsWe considered approaches to analysis of relations of events that correspond to nodes of the event net. Specifically, we proposed a method of linking live and replay scenes in the broadcasted sports video, focusing on identical and similar relations.3. Application of media organization: video abstraction and video portalBy extracting highlight events from the event net, highlight based video abstraction method was pro- posed. The experimental results demonstrated that an hour-length video can be compressed into a minute-length personalized abstract. In addition, we constructed video portal that is a tool to support our access to a multimedia space as an application of media organization.