Testing and Debugging Machine Learning-based Autonomous Systems

为了在开放的人类环境中部署自动驾驶系统，例如全自动驾驶汽车，有必要确保它们符合功能安全要求。对于集成了传统软件组件的系统，这是通过功能安全认证实践来实现的，这些实践保证了规范的有效性、实现的可理解性和实现的正确性。不幸的是，现代自主系统依赖于深度神经网络（DNN）组件，由于其黑箱性质，这些组件阻碍了传统认证实践的采用。事实上，基于DNN的系统的规格是隐式的，由用于训练DNN的输入组成；此外，它们的实现由多层神经元组成，这些神经元的逻辑无法通过检查来理解。我们将通过开发自动化解决方案来克服这些限制，以验证自主系统软件与其功能安全要求的合规性。为了实现这一目标，我们将开发创新和有效的方法来利用仿真软件。实际上，对于基于dnn的系统，模拟环境是必要的，因为规范的有效性、实现的可理解性和实现的正确性只能通过系统的运行来评估。为了有效地执行安全符合性验证活动，我们将使用进化算法驱动模拟器生成导致DNN错误的输入。为了支持问题理解，我们将利用算法来提取有关DNN行为的信息。为了改进系统，我们将使用模拟器和生成网络生成训练集。

To deploy autonomous systems, such as fully autonomous cars in open human environments, it is necessary to ensure their adherence to functional safety requirements. With systems integrating traditional software components, this is achieved through functional safety certification practices that assure specifications validity, implementation understandability, and implementation correctness. Unfortunately, modern autonomous systems rely on Deep Neural Network (DNN) components that prevent the adoption of traditional certification practices because of their black-box nature. Indeed, the specifications of DNN-based systems are implicit and consist of the inputs used to train the DNN; also, their implementation consists of layers of neurons whose logic cannot be understood through inspection. We will overcome these limitations by developing automated solutions to verify the compliance of autonomous system software with its functional safety requirements. To achieve this objective, we will develop innovative and efficient methods to leverage simulation software. Indeed, with DNN-based systems, simulated environments are necessary because specifications validity, implementation understandability, and implementation correctness can be evaluated only by exercising the system. To efficiently perform safety compliance verification activities, we will use evolutionary algorithms to drive simulators towards the generation of inputs that lead to DNN errors. To support problem understanding, we will leverage algorithms that extract information concerning DNN behaviour. To improve the system, we will generate training sets using simulators and generative networks.