Multiple Emotion Tagging for Multimedia Data by Exploiting High-Order Dependencies Among Emotions
Multiple Emotion Tagging for Multimedia Data by Exploiting High-Order Dependencies Among Emotions
复制标题
DOI：
10.1109/tmm.2015.2484966
复制
发表时间：
2015-10
影响因子：
7.3
通讯作者：
Shangfei Wang;Jun Wang;Ziheng Wang;Q. Ji
中科院分区：
文献类型：
--
作者：
Shangfei Wang;Jun Wang;Ziheng Wang;Q. Ji
本文提出了一种新的多情感多媒体标注方法，该方法显式地对情感之间的高阶关系进行建模。首先，从多媒体数据中提取多媒体特征。其次，使用传统的多标签分类器来获取多情感标签的度量。然后，我们提出了一个三层受限Boltzmann机器(TRBM)模型来捕捉情感标签之间的高阶关系，以及标签和度量之间的关系。最后，利用TRBM模型将情感度量与多情感之间的依赖关系相结合来推断样本的多情感标签。在四个数据库上的实验结果表明，我们的方法比基于特征驱动的方法和现有的基于模型的方法都更有效，后者通过贝叶斯网络(BN)来捕捉标签之间的成对关系。此外，将BN模型与TRBM模型进行了比较，验证了TRBM潜在单元捕获的模式不仅包含BN捕获的所有依赖关系，还包含BN无法捕获的其他依赖关系。
In this paper, a novel approach of multiple emotional multimedia tagging is proposed, which explicitly models the higher-order relations among emotions. First, multimedia features are extracted from the multimedia data. Second, a traditional multi-label classifier is used to obtain the measurements of the multi-emotion labels. Then, we propose a three-layer restricted Boltzmann machine (TRBM) model to capture the higher-order relations among emotion labels, as well as the relations between labels and measurements . Finally , the TRBM model is used to infer the samples' multi- emotion labels by combining the emotion measurements with the dependencies among multi- emotions . Experimental results on four databases demonstrate that our method is more effective than both feature -driven methods and current model-based methods, which capture the pairwise relations among labels by the Bayesian network (BN). Furthermore , the comparison of BN models and the proposed TRBM model verifies that the patterns captured by the latent units of TRBM contain not only all the dependencies captured by the BN but also many other dependencies that the BN cannot capture.