面向社交媒体的细粒度情感分析近几年来是自然语言处理领域的前沿问题和研究热点。作为社交媒体细粒度情感分析的两个重要子任务，实体抽取和实体级情感分类已经受到了深入研究。然而，目前大部分相关工作仍然局限于理解社交媒体上的文本内容。随着社交媒体上的内容逐渐趋于多模态化，如何结合图片、视频等多模态信息来进行实体抽取以及情感分类给细粒度文本情感分析研究带来了新的挑战。针对上述状况，本项目将开展下列三方面的工作：1）针对现有多模态实体抽取研究在多模态融合方面的缺陷，提出一个基于跨模态翻译与融合机制的实体抽取模型；2）基于1中抽取出的实体，进一步研究实体级多模态情感分类，构建一个基于层级式交互与门控融合机制的模型；3）最后针对前两个任务存在的标注匮乏问题，提出一个基于无监督学习的多模态预训练模型框架。本项目将有助于推动面向社交媒体上海量多模态内容的情感分析方法研究，对该研究的广泛应用具有潜在意义。

Fine-grained sentiment analysis of social media posts has been a frontier research direction in natural language processing in recent years. As two important sub-tasks of fine-grained sentiment analysis in social media, entity extraction and entity-level sentiment classification have been well studied in the literature. However, most existing studies are still limited to understanding the textual contents in social media. As social media posts become increasingly multimodal, how to integrate all the multimodal information for better entity extraction and sentiment analysis has become a key challenge of current fine-grained sentiment analysis research. In this project, we will focus on the following three aspects: First, motivated by the limitations of existing multimodal entity extraction studies on multimodal fusion, we plan to propose an entity extraction architecture with a novel cross-modal translation and fusion mechanism; Second, on basis of the extracted entities, we consider to design a hierarchical interaction and gated fusion mechanism for entity-level multimodal sentiment classification; Finally, we plan to propose a multimodal pre-training framework to address the label sparsity problem of the two tasks. This project will help promote the sentiment analysis research of the huge amount of multimodal posts in social media, and has potential value for its wide application in the industry.