A Human-Trustable Self-Improving Machine Learning Framework for Rapid Disaster Responses Using Satellite Sensor Imagery

近年来，由于地球气候的突变和全球城市化的急剧上升，自然灾害变得不可预测，并造成了巨大的社会和经济破坏。根据一项发表的研究，2015-2019年间，报告的自然灾害共有1624起，如地震、洪水、山体滑坡等，全球平均每年造成6万人死亡。虽然人类在大多数情况下无法预防自然灾害，但及时的反应可以在救灾和拯救生命方面发挥关键作用。在人道主义援助和灾害应对中，需要快速准确的建筑物损坏评估(BDA)来开展救生工作。然而，目前的BDA大多是基于人工检查和记录，这是耗时和劳动密集型的。尽管高分辨率卫星传感器图像，如GeoEye-1、WorldView-2和3，已成为BDA第一手资料的主要来源，但这些图像往往呈现出复杂的几何结构和空间格局。在时间紧迫的情况下，从受灾地区的HRSSIS中自动提取信息是势在必行的，并有可能促进灾后评估，加快拯救生命的进程。然而，对于最先进的机器学习(ML)算法来说，这仍然是一项极具挑战性的任务。传统的基于ML的BDA方法利用支持向量机、随机森林等主流分类器，根据从灾前和灾后图像中提取的手工特征生成破坏图。然而，HRSSIS的复杂性和异构性阻碍了传统方法的适用性，使得特征提取变得非常困难。此外，建筑物通常只涉及几个像素，留下最少的结构信息可供利用。虽然传统的方法不需要大量的训练图像，并且更容易解释，但它们在真实场景中很容易失败。另一方面，深度学习技术，特别是深度卷积神经网络(DCNN)在计算机视觉和模式识别领域取得了显著的成就。最近的一些研究探索了DCNN对BDA的作用能力，并在实验条件下报道了有希望的结果。基于DCNN的方法已经变得越来越流行，目前是BDA研究的最先进的方法。然而，DCNN通常被描述为黑盒，并且是计算密集型和数据饥渴的。由于DCNN的潜在机制与人类不同，也无法理解，DCNN在不熟悉的情况下很容易由于不确定性而失败，并且经常被观察到表现出意想不到的行为。这些缺点阻碍了基于DCNN的BDA方法在实际场景中的实际应用。因此，应急管理服务(EMSS)，例如《国际空间和重大灾害宪章》，仍然依赖对HRSSIS的目测解释来评估由于可靠性而造成的建筑物损坏。为了使基于ML的BDA方法在现实世界的场景中可靠，该项目旨在通过开发高度创新的可解释ML(XML)技术来自动化基于灾后HRSSIS的BDA过程，从而催化人工智能领域的一步变化。开发的XML技术将作为场景理解、建筑物分割和损害评估的框架，在场景级和像素级联合进行，并具有实时自适应不同应用场景的能力，以应对现实世界的不确定性。通过实现可靠的自动化解决方案来促进具有高度挑战性的灾后BDA任务，我们的最终目标是帮助EMSS更快地进行灾后评估，促进救生过程。

Due to the abrupt changes in Earth's climate and the dramatic global rise of urbanisation, natural disasters have become unpredictable and caused great social and economic devastation in recent years. According to one published study, between 2015-2019, there were a total of 1624 reported natural disasters, such as earthquakes, floods, landslides, etc., killing on average 60,000 people each year globally. Although humans cannot prevent natural disasters in most cases, timely responses can play a critical role in disaster relief and life-saving. Rapid and accurate building damage assessment (BDA) is required in humanitarian assistance and disaster response to carry out life-saving efforts. However, current BDAs are mostly based on manual inspection and documentation, which is time consuming and labour-intensive. Although high-resolution satellite sensor images (HRSSIs) such as GeoEye-1, WorldView-2 and 3, have become the major source of first-hand information for BDA, those images often present a mosaic of complex geometrical structures and spatial patterns. Automatic information extraction from HRSSIs of disaster-affected areas is imperative under time-critical situations, and has the potential to facilitate post-disaster assessment, speed up the life-saving rescue processes. However, this remains an extremely challenging task for the state-of-the-art machine learning (ML) algorithms. In practice, human experts have to manually interpret and examine the captured HRSSIs, which involves significant time and labour.Conventional ML-based BDA methods leverage mainstream classifiers, such as support vector machine, random forest, to generate a damage map based on hand-crafted features extracted from pre- and post- disaster images. However, the complexity and heterogeneity of HRSSIs hinder the applicability of conventional methods, making feature extraction extremely difficult. Besides, buildings often involve only a few pixels, leaving minimal structural information to exploit. Although conventional methods do not require a large volume of training images and are more interpretable, they fail easily on real scenes. On the other hand, deep learning techniques, particularly, deep convolutional neural networks (DCNNs) have reported significant achievements in the field of computer vision and pattern recognition. Some recent studies have explored the capability of DCNNs on BDA and reported promising outcomes under experimental conditions. DCNN-based methods have become increasingly popular and are currently the state-of-the-art in BDA research. However, DCNNs are often characterised as black boxes, and are computationally intensive and data-hungry. As the underlying mechanisms are different from humans and not understandable, DCNNs can fail easily in unfamiliar scenarios due to uncertainties and are often observed to exhibit unexpected behaviours. These disadvantages hinder the practical utilities of DCNN-based BDA methods in real-world scenarios. As a result, emergency management services (EMSs), e.g., the International Charter Space and Major Disasters, still rely on visual interpretation of HRSSIs to assess building damage due to the reliability. To make ML-based BDA methods reliable for real-world scenarios, this project aims to catalyse a step-change in artificial intelligence by developing highly innovative explainable ML (XML) techniques to automate the BDA processes based on post-disaster HRSSIs. The developed XML techniques will act as a framework for scene understanding, building segmentation and damage assessment on both scene-level and pixel-level in a joint fashion, and have the capacity to self-adapt to different application scenarios in real-time to address real-world uncertainties. By achieving a reliable automated solution to facilitate the highly challenging post-disaster BDA task, we ultimately aim to assist EMSs for faster post-disaster assessment, facilitating life-saving process.