Multimedia Summarization for Social Events in Microblog Stream
Multimedia Summarization for Social Events in Microblog Stream
复制标题
DOI：
10.1109/tmm.2014.2384912
复制
发表时间：
2015-02-01
影响因子：
7.3
通讯作者：
Chua, Tat-Seng
中科院分区：
文献类型：
--
作者：
Bian, Jingwen;Yang, Yang;Chua, Tat-Seng
微博服务彻底改变了人们交换信息的方式。面对日益增多的社交事件以及相应的多媒体内容的微博，需要提供可视化的摘要来帮助用户快速掌握这些社交事件的本质，以便更好地理解。虽然现有的方法大多只关注基于文本的摘要，但很少探索具有多种媒体类型（例如文本、图像和视频）的微博摘要。在本文中，我们提出了一种多媒体社交事件摘要框架，可以从多种媒体类型的微博流中自动生成可视化摘要。具体来说，所提出的框架包括三个阶段，如下所示。 1）首先设计噪声去除方法来消除潜在的噪声图像。利用有效的光谱过滤模型来估计图像与给定事件相关的概率。 2）提出了一种新的跨媒体概率模型，称为跨媒体LDA（CMLDA），用于联合发现多种媒体类型的微博中的子事件。这些不同媒体类型之间的内在相关性得到了很好的探索和利用，以加强跨媒体子事件发现过程。 3）最后，基于所有发现的子事件的跨媒体知识，设计了多媒体微博摘要生成过程，以联合识别代表性文本和视觉样本，并进一步聚合以形成整体的可视化摘要。我们对两个现实世界的微博数据集进行了广泛的实验，以证明所提出的框架与最先进的方法相比的优越性。
Microblogging services have revolutionized the way people exchange information. Confronted with the ever-increasing numbers of social events and the corresponding microblogs with multimedia contents, it is desirable to provide visualized summaries to help users to quickly grasp the essence of these social events for better understanding. While existing approaches mostly focus only on text-based summary, microblog summarization with multiple media types (e.g., text, image, and video) is scarcely explored. In this paper, we propose a multimedia social event summarization framework to automatically generate visualized summaries from the microblog stream of multiple media types. Specifically, the proposed framework comprises three stages, as follows. 1) A noise removal approach is first devised to eliminate potentially noisy images. An effective spectral filtering model is exploited to estimate the probability that an image is relevant to a given event. 2) A novel cross-media probabilistic model, termed Cross-Media-LDA (CMLDA), is proposed to jointly discover subevents from microblogs of multiple media types. The intrinsic correlations among these different media types are well explored and exploited for reinforcing the cross-media subevent discovery process. 3) Finally, based on the cross-media knowledge of all the discovered subevents, a multimedia microblog summary generation process is designed to jointly identify both representative textual and visual samples, which are further aggregated to form a holistic visualized summary. We conduct extensive experiments on two real-world microblog datasets to demonstrate the superiority of the proposed framework as compared to the state-of-the-art approaches.