Improving Neural Network Explainability for Time Series through in-distribution obfuscation strategies

给定一个神经网络，在没有关于它是如何训练的先验知识的情况下，我们想分析它如何做出决策。从这个意义上说，可解释性是关于打开黑盒子，确定输入数据的哪些关键特征有助于某个输出。神经网络的可解释性在涉及用户的设置中很重要，例如用户申请职位或获得医疗评估的应用设置，并由基于深度学习技术的网络处理。我们的目标是从算法中提取为什么会达到这个特定的应用决策或医疗诊断，从而提高模型的透明度和可信度。可解释的人工智能还可以帮助科学家获得更好的反馈，以进行改进，并获得有关其数据的见解。可解释性通常与图像相关，因为特征与像素之间存在直观的对应关系。其目的是将可解释性扩展到其他模式，例如时间序列，这些模式一直存在于深度学习技术中，并与许多领域相关，包括医疗和金融。这种扩展需要对特征相关性的定义进行概括，这是通过率失真理论来完成的。为了将相关性分配给特征的更复杂的导数，我们使用时间序列的适当分解，例如傅立叶变换。然后选择一个小的相关分量，并使用遵循扰动分布的随机噪声向量对其余数据进行混淆。在定义适合于分类的失真度量之后，我们可以评估混淆掩码在随机噪声向量上的预期失真。因此，找到最佳掩模成为受限于掩模的大小的最小化问题。我们的目标是生成一个混淆，最大限度地减少失真，同时保持相关组件在一定的大小。这个最佳相关分量就是对决策的解释。改进这个方法需要从几何学的角度来看待这个过程，其中这个混淆掩码可以被解释为在模型映射下稳定的子空间。这意味着在噪声矢量遵循扰动分布的情况下，在该掩模的模型下失真较低。创新之处在于为时间序列选择正确的表示，以便在新坐标下最大限度地减少所选掩码的失真。除了数据表示之外，解释的意义在很大程度上取决于扰动分布的性质。我们的目标是选择这种随机向量的分布，使混淆的数据位于模型训练的自然数据流形上，因为在这个流形之外可能存在模型的未开发区域，这可能导致解释错误。为了实现这一点，我们使用了一个生成对抗网络，在这个网络中，我们训练一个生成器，使得生成的数据与训练集中的数据根据一个训练集无法区分。这样，生成的数据位于期望的数据流形上，并且可以基于生成器的潜变量来选择扰动器分布。因此，通过使混淆不是完全随机的，而是使它们类似于真实世界的时间序列，这有望通过减少损坏的解释来实现提高可解释性。该项目属于EPSRC人工智能技术研究领域的福尔斯。

Given a Neural Network, and without prior knowledge of how it was trained, we want to analyze how it reaches decisions. In this sense, explainability is about opening the black box and determining what are the key features of the input data that contribute to a certain output.Explainability of Neural Networks is important in settings where a user is involved, such as an application setting where users apply for a position or get a medical evaluation and that is processed by a network based on deep learning techniques. The goal is to extract from the algorithm why this particular application decision or medical diagnosis was reached, and thus improving transparency and trustworthiness of the model. Explainable AI can also help scientists get better feedback to make improvements as well as get insights about their data.Explainability has most often been associated with images, due to the intuitive correspondence of features to pixels. The aim is to extend explainability to other modalities, such as time series, which are ever present in Deep Learning techniques and associated with many fields, including medical and financial. This expansion requires a generalization on the definition of feature relevance, and this is done through Rate-Distortion theory. In order to assign relevance to more complex derivatives of features, we use appropriate decompositions of time series, such as the Fourier transform. Then a small relevant component is chosen, and the rest of the data is obfuscated using a random noise vector that follows a perturbation distribution. After defining a distortion metric appropriate for classification, we can evaluate the expected distortion of the obfuscation mask over the random noise vectors. Finding the best mask therefore becomes a minimization problem subject to limiting the size of the mask. The goal is to generate an obfuscation that minimizes distortion while keeping the relevant component under a certain size. This optimal relevant component is then the explanation for the decision.Improving upon this method requires viewing this procedure from a geometric viewpoint, where this obfuscation mask can be interpreted as a subspace that is stable under the model's mapping. This means that distortion is low under the model for that mask given the noise vector follows the perturbator distribution. The innovation comes in selecting the right representation for time series so that distortion is minimized for selected masks under the new coordinates. In addition to the data representation, the meaning of an explanation greatly depends on the nature of the perturbation distribution. The goal is to select this distribution of random vectors so that the obfuscated data lies on the natural data manifold that the model was trained on, because outside this manifold may lie undeveloped regions of the model, which can result in corrupted explanations. In order to achieve this, we use a generative adversarial network, where we train a generator in such a way that the generated data is indistinguishable from the data in the training set according to a discriminator. This way, the generated data lies on the desired data manifold, and the choice of perturbator distribution can be made based on the latent variables of the generator. Therefore, by making obfuscations not entirely random, but instead have them resemble real-world time series, which is expected to achieve improved explainability by reducing corrupted explanations.This project falls within the EPSRC Artificial Intelligence Technologies research area.