权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Systemization of audio-visual knowledge resources using graphical models

利用图模型将视听知识资源系统化

基本信息

批准号：
17300059
负责人：
SHINODA Koichi
金额：
$ 9.46万
依托单位：
Tokyo Institute of Technology
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (B)
财政年份：
2005
资助国家：
日本
起止时间：
2005 至 2007
项目状态：
已结题

项目摘要

Recent advances in computer technology, particularly in storage technology, have resulted in significant increases in the number and quality of audio-visual knowledge resources. Most of those resources are not equipped with index information, and thus, it has become difficult for ordinary people to browse the entire content of each database. Techniques for systemizing audio-visual knowledge resources and utilizing them have been strongly demanded. However, statistical pattern recognition techniques have not yet achieved enough performance for this purpose. In addition, it is not always clear what kinds of indexing are useful. In this study, we take an approach to index those databases in different ways with unsupervised manner, and extract dependencies among those labels. First, we carried scene recognition for baseball video. We constructed annotated database for 43 games of Major League Baseball with NHK Science & Technical Research Labs and used them for our evaluation. We used vari … More ous relationships between scene labels such as scene contexts, and unified audio and visual information. We achieved 60% accuracy for 16 scene recognition and 90% recall rate for score scene detection. Our techniques are expected to contribute much to make automatic highlight extraction systems for broadcast companies. Second, we participated in TRECVID workshop organized by NIST, USA, to study the high-level feature extraction task. We constructed tree-structured dictionaries of "visual words" by unsupervised clustering for video features, and selected a tree-cut as a dictionary for each word. By using Bag-of-word approach, we constructed a robust extraction system against the differences in data amount for each feature. We also extracted effective "motion words" for dynamic features. Our method achieved significant improvements in the task of extracting 39 features. The other research topics include robust speech recognition using graphical models, multi-modal interface for asynchronous multi-modal inputs, human-gait modeling. Less

最近计算机技术，特别是存储技术的进步，使视听知识资源的数量和质量都有了显著提高。这些资源大多没有配备索引信息，因此，普通人很难浏览每个数据库的全部内容。对视听知识资源的系统化和利用技术提出了强烈的要求。然而，统计模式识别技术在这方面还没有达到足够的性能。此外，我们并不总是清楚哪些索引是有用的。在本研究中，我们采用无监督的方式对这些数据库进行不同方式的索引，并提取这些标签之间的依赖关系。首先，我们对棒球视频进行了场景识别。我们与NHK科学技术研究实验室建立了美国职业棒球大联盟43场比赛的注释数据库，并使用它们进行评估。我们使用了场景标签（如场景上下文）之间的各种关系，以及统一的音频和视觉信息。我们对16个场景识别的准确率达到60%，对分数场景检测的召回率达到90%。我们的技术有望为广播公司的自动高光提取系统做出贡献。其次，我们参加了美国NIST组织的TRECVID研讨会，研究高级特征提取任务。通过对视频特征的无监督聚类，构建了“视觉词”的树状字典，并为每个词选择了一个树切作为字典。通过使用词袋方法，我们针对每个特征的数据量差异构建了一个鲁棒的提取系统。我们还为动态特征提取了有效的“运动词”。我们的方法在提取39个特征的任务上取得了显著的改进。其他研究课题包括使用图形模型的鲁棒语音识别，异步多模态输入的多模态界面，人体步态建模。少