Testing and Debugging Machine Learning-based Autonomous Systems

为了在开放的人类环境中部署自主系统，例如完全自主的汽车，有必要确保它们符合功能安全要求。对于集成传统软件组件的系统，这是通过功能安全认证实践来实现的，这些实践确保了规范的有效性、实现的可理解性和实现的正确性。不幸的是，现代自治系统依赖于深度神经网络（DNN）组件，由于其黑箱性质，这些组件阻止了传统认证实践的采用。事实上，基于DNN的系统的规范是隐式的，由用于训练DNN的输入组成;此外，它们的实现由神经元层组成，其逻辑无法通过检查来理解。我们将通过开发自动化解决方案来克服这些限制，以验证自主系统软件是否符合其功能安全要求。为了实现这一目标，我们将开发创新和有效的方法来利用模拟软件。事实上，对于基于DNN的系统，模拟环境是必要的，因为规范的有效性，实现的可理解性和实现的正确性只能通过测试系统来评估。为了有效地执行安全合规性验证活动，我们将使用进化算法来驱动模拟器生成导致DNN错误的输入。为了支持问题理解，我们将利用提取有关DNN行为信息的算法。为了改进系统，我们将使用模拟器和生成网络生成训练集。

To deploy autonomous systems, such as fully autonomous cars in open human environments, it is necessary to ensure their adherence to functional safety requirements. With systems integrating traditional software components, this is achieved through functional safety certification practices that assure specifications validity, implementation understandability, and implementation correctness. Unfortunately, modern autonomous systems rely on Deep Neural Network (DNN) components that prevent the adoption of traditional certification practices because of their black-box nature. Indeed, the specifications of DNN-based systems are implicit and consist of the inputs used to train the DNN; also, their implementation consists of layers of neurons whose logic cannot be understood through inspection. We will overcome these limitations by developing automated solutions to verify the compliance of autonomous system software with its functional safety requirements. To achieve this objective, we will develop innovative and efficient methods to leverage simulation software. Indeed, with DNN-based systems, simulated environments are necessary because specifications validity, implementation understandability, and implementation correctness can be evaluated only by exercising the system. To efficiently perform safety compliance verification activities, we will use evolutionary algorithms to drive simulators towards the generation of inputs that lead to DNN errors. To support problem understanding, we will leverage algorithms that extract information concerning DNN behaviour. To improve the system, we will generate training sets using simulators and generative networks.