融合跨模态深度分析与迭代优化的图像描述自动生成方法研究

本项目针对跨模态图像描述生成方法进行深入研究，探索、建立和完善面向大规模多模态信息源并融合跨模态深度分析与迭代优化机制的图像描述自动生成基本理论、方法和实现。通过融合多模态信息源的深度理解、分析与优化关键技术，建立有效适用的图像描述自动生成机制，为图像视觉内容的深层次理解提供高效、精确、易用的获取合理语义信息的智能支持。其中，采取图像多模态内容展现形式作为重要基础信息源，有效利用图像视觉信息与文本描述的多模态深度特征属性表征与融合、跨模态深度主题关联分析与学习、跨模态深度语义相关性学习与融合、融合多模态注意力的跨模态描述生成建模、及融合迭代优化的跨模态深度精炼与优选等多种跨模态深层次分析、处理与优化机制，并针对面向跨模态描述生成所需的深度神经网络与多模态注意力相关理论与工作机制进行深入探索，以期构成功能更加强壮且性能更加完善的图像描述自动生成架构，形成视觉内容描述跨模态自动生成的新形态。

This project focuses on the research for image captioning with cross-modal deep analysis and iterative optimization, so as to explore, establish and improve the basic theory, approach and implementation for image captioning that fuses large-scale multimodal information sources, cross-modal deep analysis and iterative optimization mechanisms. It aims to construct a more effective and appropriate image captioning scheme by integrating the deep understanding, analysis and optimization techniques for multimodal information sources. Thus the intelligent support for efficient, precise and easy-to-use acquirement of reasonable semantic information can be provided for deep-level visual content understanding. Visual information users can promote their efficiency for obtaining beneficial and valuable information from large-scale image data. Multiple new key techniques for image captioning are deeply studied and developed to facilitate more effective caption generation with the stronger function and more perfect performance. They focus on multiple cross-modal deep-level analysis, processing and optimization mechanisms, including multimodal deep feature attribute representation and fusion, cross-modal deep topic association analysis and learning, cross-modal deep semantic correlation learning and fusion, cross-modal captioning modeling with multimodal attention, and cross-modal deep refinement and optimization with iterative learning, etc. Meanwhile, the related theory and working mechanisms for deep neural network and multimodal attention are also deeply explored. All these research work aims to establish a stronger and better cross-modal image captioning architecture, and construct a new form for visual content captioning.