深度学习是目前处理大数据的主流方法，然而这种方法并未考虑到大数据的多模态性质，难以实现准确应用决策，且学习网络不能灵活应对应用场景的不断变化。针对这两点不足，本项目提出面向多模态大数据的演进式深度增强学习方法，具体包括：研究从像素域和测量域上提取图像/视频数据多模态特征的方法，并选取能够匹配应用需求的特征；研究基于模态间相关性和权重更新的多模态特征融合方法，用于适配深度增强学习网络；基于应用驱动研究演进式深度增强学习算法，完成网络节点权重和网络结构演进；研究基于应用场景反馈的多模态特征融合优化算法，进而实现网络和融合特征的联合优化，最终以最小的代价满足应用精确度要求。本项目的研究通过挖掘模态间内在相关性得到多模态特征的最优融合，进而基于融合特征，针对不同应用需求演进优化学习网络，获得最优的行为策略，从而更加准确、智能地理解以图像/视频为主的多模态大数据，推动多媒体技术与人工智能的发展。

Deep learning seems to be the most popular method to deal with big data problems. Two main drawbacks, however, still exist. Lack of considering multimodal data will reduce the accuracy of decision making, and the networks are not flexible enough to handle variety of applications. Therefore, we propose evolutionary deep reinforcement learning targeting multimodal big data, which includes studying extraction methods for multimodal features of image/video data in terms of the pixel domain and the measurement domain respectively, then further satisfying the requirements of applications; studying fusion methods for multimodal big data based on their weights and correlations to match the networks; studying weights and architecture evolutionary methods for evolutionary deep reinforcement learning networks based on applications, and studying optimization methods for multimodal big data fusion, so that both networks and fusion methods can be jointly optimized to maximize the accuracy with minimum cost. This project aims to obtain optimized fused multimodal features via discovering the coherent correlations between modals, so that the networks can evolutionally optimize their weights and architecture, in order to achieve the dominating policy to satisfy the requirements of application. This project will provide more accurate and intelligent comprehension of image/video-based multimodal big data, and further to facilitate the development of multimedia technology and artificial intelligence.