Multi-attention Recurrent Network for Human Communication Comprehension
Multi-attention Recurrent Network for Human Communication Comprehension
复制标题
DOI：
10.1609/aaai.v32i1.12024
复制
发表时间：
2018-02
期刊：
影响因子：
--
通讯作者：
Amir Zadeh;P. Liang;Soujanya Poria;Prateek Vij;E. Cambria;Louis-Philippe Morency
中科院分区：
文献类型：
--
作者：
Amir Zadeh;P. Liang;Soujanya Poria;Prateek Vij;E. Cambria;Louis-Philippe Morency
人类的面对面交流是一种复杂的多模态信号。我们使用词语（语言模态）、手势（视觉模态）和语调的变化（声学模态）来传达我们的意图。人类很容易处理和理解面对面的交流，然而，理解这种形式的交流仍然是人工智能（AI）的一个重大挑战。人工智能必须理解每一种模态以及它们之间的交互作用，从而形成通信。在本文中，我们提出了一种新的神经结构，用于理解人类的通信称为多注意递归网络（MARN）。我们模型的主要优势来自于使用称为多注意力块（MAB）的神经组件通过时间发现模态之间的相互作用，并将其存储在称为长短期混合记忆（LSTHM）的循环组件的混合记忆中。我们对六个公开的数据集进行了广泛的比较，用于多模态情感分析，说话人特征识别和情感识别。MARN在所有数据集中显示了最先进的结果性能。
Human face-to-face communication is a complex multimodal signal. We use words (language modality), gestures (vision modality) and changes in tone (acoustic modality) to convey our intentions. Humans easily process and understand face-to-face communication, however, comprehending this form of communication remains a significant challenge for Artificial Intelligence (AI). AI must understand each modality and the interactions between them that shape the communication. In this paper, we present a novel neural architecture for understanding human communication called the Multi-attention Recurrent Network (MARN). The main strength of our model comes from discovering interactions between modalities through time using a neural component called the Multi-attention Block (MAB) and storing them in the hybrid memory of a recurrent component called the Long-short Term Hybrid Memory (LSTHM). We perform extensive comparisons on six publicly available datasets for multimodal sentiment analysis, speaker trait recognition and emotion recognition. MARN shows state-of-the-art results performance in all the datasets.