基于高清光学遥感图像的城市场景理解在城市规划、灾害监测等领域具有重大的应用价值。高清光学遥感图像中同类目标间外观特征差异巨大和目标严重交叠遮挡为城市场景理解带来巨大挑战。为了解决上述挑战性问题，本项目开展基于与或图和多尺度卷积网络的城市场景理解方法研究。针对同类目标间外观特征差异大的问题，融合与或图模型（集合分层图结构、属性语法、概率模型）和多尺度卷积网络（提取多尺度深层特征）的优势，实现对各类场景及之间关系、目标及之间关系的准确描述，尤其是实现对外观特征差异大的同类目标的有效描述。针对目标严重交叠遮挡问题，基于分层图结构，自下而上地组合局部特征以推理全局场景，同时自上而下地利用全局特征预测局部信息，融合上述双向推理以修正场景理解结果，实现对城市场景的准确理解，并能很好地解释目标交叠遮挡关系并预测目标被遮挡部分。最后，将本项目方法初步应用于城市用地现状统计、建筑物密度图计算、道路网构建等。

Urban scene understanding based on high-resolution visible remote sensing images has a significant application value in various applications, such as urban planning and disaster monitoring. Diverse appearance patterns among object instances and serious overlaps and occlusions among objects present great challenges for urban scene understanding. To solve these problems, this proposal will propose a method based on And-Or-Graph (AOG) and Multi-scale Convolutional Network (MCN) for urban scene understanding from high-resolution visible remote sensing images. To handle the diverse appearance patterns among object instances, the AOG and MCN will be fused to represent urban scenes. The advantages of AOG and MCN are that AOG gathers the hierarchical graph structure, attribute grammar, probability model, and MCN can extract multi-scale deep features for global scenes and local objects. Therefore, the fusion of the AOG and MCN can well describe various scenes, objects, and relationships among them, especially the object instances with diverse appearance patterns. To solve the serious overlaps and occlusions among objects, a bottom-up inference and a top-down inference will be fused to parsing the urban scene. In the bottom-up inference, based on the hierarchical graph structure, the local features will be combined to infer the global scene. Simultaneously, the global features will be utilized to predict local information in the top-down inference. Then, the results of the two inferences will be fused to modify the results of scene understanding. The fusion of the bidirectional inferences will achieve correct parsing of the urban scene, and especially well interpret the overlap relation and occlusion relation among objects and predict the occluded parts for objects. Finally, the proposed method will be applied in analyzing current situation of urban land use, computing building density map, recognizing road network, etc.