Topology Dictionary for 3D Video Understanding
Topology Dictionary for 3D Video Understanding
复制标题
DOI：
10.1109/tpami.2011.258
复制
发表时间：
2012-08
影响因子：
23.6
通讯作者：
Tony Tung;T. Matsuyama
中科院分区：
文献类型：
--
作者：
Tony Tung;T. Matsuyama
本文提出了一种实现三维视频理解的新方法。3D视频由运动中的主体的3D模型流组成。获取长序列需要较大的存储空间（1分钟2 GB）。此外，浏览数据集和提取有意义的信息是繁琐的。我们提出了拓扑字典来编码和描述3D视频内容。该模型由一个基于拓扑的形状描述符字典组成，该字典可以从提取的模式或训练序列中生成。该模型依赖于1)使用Reeb图进行拓扑描述和分类，2)使用马尔可夫运动图表示拓扑变化状态。我们表明，使用Reeb图作为高级拓扑描述符是相关的。它允许字典自动对复杂序列建模，而其他策略则需要事先了解捕获主题的形状和拓扑结构。该方法用于对3D视频序列进行编码，可用于基于内容的3D视频序列描述和摘要。此外，学习过程中的拓扑类标记使系统能够执行基于内容的事件识别。在不同的3D视频上进行了实验。我们展示了一个使用拓扑字典进行3D视频累进摘要的应用程序。
This paper presents a novel approach that achieves 3D video understanding. 3D video consists of a stream of 3D models of subjects in motion. The acquisition of long sequences requires large storage space (2 GB for 1 min). Moreover, it is tedious to browse data sets and extract meaningful information. We propose the topology dictionary to encode and describe 3D video content. The model consists of a topology-based shape descriptor dictionary which can be generated from either extracted patterns or training sequences. The model relies on 1) topology description and classification using Reeb graphs, and 2) a Markov motion graph to represent topology change states. We show that the use of Reeb graphs as the high-level topology descriptor is relevant. It allows the dictionary to automatically model complex sequences, whereas other strategies would require prior knowledge on the shape and topology of the captured subjects. Our approach serves to encode 3D video sequences, and can be applied for content-based description and summarization of 3D video sequences. Furthermore, topology class labeling during a learning process enables the system to perform content-based event recognition. Experiments were carried out on various 3D videos. We showcase an application for 3D video progressive summarization using the topology dictionary.