Improving Neural Network Explainability for Time Series through in-distribution obfuscation strategies

点击翻译按钮获取中文摘要

Given a Neural Network, and without prior knowledge of how it was trained, we want to analyze how it reaches decisions. In this sense, explainability is about opening the black box and determining what are the key features of the input data that contribute to a certain output.Explainability of Neural Networks is important in settings where a user is involved, such as an application setting where users apply for a position or get a medical evaluation and that is processed by a network based on deep learning techniques. The goal is to extract from the algorithm why this particular application decision or medical diagnosis was reached, and thus improving transparency and trustworthiness of the model. Explainable AI can also help scientists get better feedback to make improvements as well as get insights about their data.Explainability has most often been associated with images, due to the intuitive correspondence of features to pixels. The aim is to extend explainability to other modalities, such as time series, which are ever present in Deep Learning techniques and associated with many fields, including medical and financial. This expansion requires a generalization on the definition of feature relevance, and this is done through Rate-Distortion theory. In order to assign relevance to more complex derivatives of features, we use appropriate decompositions of time series, such as the Fourier transform. Then a small relevant component is chosen, and the rest of the data is obfuscated using a random noise vector that follows a perturbation distribution. After defining a distortion metric appropriate for classification, we can evaluate the expected distortion of the obfuscation mask over the random noise vectors. Finding the best mask therefore becomes a minimization problem subject to limiting the size of the mask. The goal is to generate an obfuscation that minimizes distortion while keeping the relevant component under a certain size. This optimal relevant component is then the explanation for the decision.Improving upon this method requires viewing this procedure from a geometric viewpoint, where this obfuscation mask can be interpreted as a subspace that is stable under the model's mapping. This means that distortion is low under the model for that mask given the noise vector follows the perturbator distribution. The innovation comes in selecting the right representation for time series so that distortion is minimized for selected masks under the new coordinates. In addition to the data representation, the meaning of an explanation greatly depends on the nature of the perturbation distribution. The goal is to select this distribution of random vectors so that the obfuscated data lies on the natural data manifold that the model was trained on, because outside this manifold may lie undeveloped regions of the model, which can result in corrupted explanations. In order to achieve this, we use a generative adversarial network, where we train a generator in such a way that the generated data is indistinguishable from the data in the training set according to a discriminator. This way, the generated data lies on the desired data manifold, and the choice of perturbator distribution can be made based on the latent variables of the generator. Therefore, by making obfuscations not entirely random, but instead have them resemble real-world time series, which is expected to achieve improved explainability by reducing corrupted explanations.This project falls within the EPSRC Artificial Intelligence Technologies research area.