基于多模态深度学习的遥感图像场景分类方法研究

遥感图像场景分类是高分辨率遥感图像解译中的关键环节，现有方法在特征描述和场景类别之间仍存在严重的“语义鸿沟”问题。本项目拟考虑图像和标签两种模态的数据，从数据表示和分类映射等多个方面入手，对高分遥感图像的场景分类与应用展开研究：首先针对遥感场景图像特征复杂异质的特点，研究顾及场景复杂度和语义层次差异的自适应深度特征提取方法，使特征能够有效表达场景图像；其次从场景标签入手，研究顾及遥感场景语义层次的标签重嵌入方法，将类别信息表现为能体现语义关系的向量；然后针对场景图像之间类内多样性大且类间相似度高的问题，研究基于语义标签监督的深度度量学习场景分类方法，建立两种模态数据表达之间的有效映射；最后研究同源和异源数据下不同场景类别的推广方法，以提升模型的泛化能力和适用性。本项目有望缩小困扰场景分类研究的语义鸿沟，最终提出一套自适应、高精度的遥感场景分类模型，为遥感图像解译与应用提供方法和技术支持。

Remote sensing image scene classification is a vital problem in the interpretation of high-resolution remote sensing images. However, the serious "semantic gap" problem between feature descriptions and semantic scene categories still remains to be solved in the existing methods. This project intends to study data expression and classification mapping based on multimodal data including images and labels, and conduct researches on the scene classification and applications of high-resolution remote sensing images. The main research contents include: (1) First, in order to extract the complex and heterogeneous features of remote sensing scenes, an adaptive deep feature extraction method with consideration of scene complexity and semantic level differences is proposed, which enables the features to effectively represent the scene contents. (2) Second, for scene labels, label re-embedding method that takes semantic levels of scene categories into account is presented to guarantee semantic relationships of category label vectors. (3) Third, in order to solve the problem of large intra-class diversity and high inter-class similarity between scene images, scene classification method based on deep metric learning and semantic label supervision is studied to establish an effective mapping of features to semantic labels. (4) Finally, applications of the methods on heterologous data and various categories are studied to improve the generalization ability of the model. This research is expected to narrow the semantic gap in the study of scene classification, and finally propose a set of adaptive and high-precision theoretical models for remote sensing scene classification, which will eventually contribute to the interpretation and application of remote sensing images.