Group Dynamics and Multimodal Interaction Modeling Using a Smart Digital Signage
Group Dynamics and Multimodal Interaction Modeling Using a Smart Digital Signage
复制标题
DOI：
10.1007/978-3-642-33863-2_36
复制
发表时间：
2012-10
期刊：
影响因子：
--
通讯作者：
Tony Tung;R. Gomez;Tatsuya Kawahara;T. Matsuyama
中科院分区：
文献类型：
--
作者：
Tony Tung;R. Gomez;Tatsuya Kawahara;T. Matsuyama
本文提出了一种新的多模态系统的群体动力学和互动分析。该框架是由一个麦克风阵列和多视图视频摄像机放置在数字标牌显示器，作为互动的支持。我们发现，视觉信息处理可以用来本地化非言语交际事件，并与音频信息同步。我们的贡献是双重的：1）我们提出了一个可扩展的便携式多人多模态交互传感系统，2）我们提出了一个通用的框架来模拟A/V多模态交互，采用扬声器日记的音频处理和混合动态系统（HDS）的视频处理。HDS通过捕捉头部运动中的时间结构特征来表示多人之间的通信动态。实验结果显示了真实世界的情况下，组通信处理的联合注意估计。我们相信，所提出的框架是非常有前途的进一步研究。
This paper presents a new multimodal system for group dynamics and interaction analysis. The framework is composed of a mic array and multiview video cameras placed on a digital signage display which serves as a support for interaction. We show that visual information processing can be used to localize nonverbal communication events and synchronized with audio information. Our contribution is twofold: 1) we present a scalable portable system for multiple people multimodal interaction sensing, and 2) we propose a general framework to model A/V multimodal interaction that employs speaker diarization for audio processing and hybrid dynamical systems (HDS) for video processing. HDS are used to represent communication dynamics between multiple people by capturing the characteristics of temporal structures in head motions. Experimental results show real-world situations of group communication processing for joint attention estimation. We believe the proposed framework is very promising for further research.