Inflation-Deflation Networks for Recognizing Head-Movement Functions in Face-to-Face Conversations
Inflation-Deflation Networks for Recognizing Head-Movement Functions in Face-to-Face Conversations
复制标题
DOI：
10.1145/3462244.3482856
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
K. Takeda;K. Otsuka
中科院分区：
文献类型：
--
作者：
K. Takeda;K. Otsuka
在面对面的交谈中，头部的运动有各种各样的功能。最近，卷积神经网络（CNN）已被提出来识别由头部运动从时间序列的对话者的头部姿势角度在多方对话期间执行的通信功能。然而，在识别性能方面还有改进的空间。为了提高CNN的性能，本文提出了一个特征膨胀-收缩模块（I/DeF模块）作为附加在CNN输入层之前的附加模块，以促进头部运动动态的特征学习。I/DeF模块由重复的充气和放气过程组成。膨胀过程通过转置卷积来放大和外推窗口化的输入时间序列。收缩过程压缩膨胀的数据并恢复其原始数据长度。针对10个频繁的头部运动功能，实验表明，具有I/DeF模块的CNN（I/DeF-CNN）在所有功能类别中的表现都优于以前的CNN，F-测量高达4.5点。我们还将I/DEF模块集成到VGG和ResNet中。与这些方法的比较表明，I/DeF-CNN在10个功能中的8个功能上超过了其他模型。这些结果证实了I/DEF模块的有效性及其在推进非言语行为识别方面的潜力。
Head movements have various functions in face-to-face conversations. Recently, convolutional neural networks (CNNs) have been proposed to recognize the communicative functions performed by the head movements from the time series of interlocutors’ head pose angles during multiparty conversations. However, there is room for improvement in the recognition performance. To boost the CNNs’ performance, this paper proposes a feature Inflation-Deflation module (I/DeF module) as an additional module attached ahead of the CNNs’ input layer to facilitate the feature learning of the head-movement dynamics. The I/DeF module consists of repeated inflation and deflation processes. The inflation process upscales and extrapolates the windowed input time series by a transposed convolution. The deflation process compresses the inflated data and recovers its original data length. Targeting the ten frequent head-movement functions, the experiments showed that CNNs with the I/DeF module (I/DeF-CNNs) outperformed the previous CNNs in all function categories up to 4.5 points in F-measure. We also integrated the I/DeF module into VGG and ResNet. Comparison to these methods showed that I/DeF-CNNs surpassed the other models for 8 out of 10 functions. These results confirmed the effectiveness of the I/DeF module and its potential for advancing nonverbal behavior recognition.