Improving Neural Network Explainability for Time Series through in-distribution obfuscation strategies

给定一个神经网络，并且事先不知道它是如何训练的，我们想要分析它是如何做出决策的。在这个意义上，可解释性是指打开黑匣子，确定输入数据的哪些关键特征对特定输出有贡献。神经网络的可解释性在用户参与的环境中非常重要，例如用户申请职位或获得医疗评估的应用程序环境，该环境由基于深度学习技术的网络处理。目标是从算法中提取为什么会达到这个特定的应用决定或医疗诊断，从而提高模型的透明度和可信度。可解释人工智能还可以帮助科学家获得更好的反馈，以进行改进，以及对他们的数据进行洞察。由于特征与像素的直观对应，可解释性通常与图像联系在一起。其目的是将可解释性扩展到其他模式，如时间序列，这些模式一直存在于深度学习技术中，并与许多领域相关，包括医疗和金融。这种扩展需要对特征相关性的定义进行泛化，这是通过率失真理论来完成的。为了给更复杂的特征导数赋予相关性，我们使用了适当的时间序列分解，例如傅里叶变换。然后选择一个小的相关分量，并使用遵循扰动分布的随机噪声向量对其余数据进行模糊处理。在定义了适合于分类的失真度量之后，我们可以评估模糊掩模在随机噪声向量上的期望失真。因此，寻找最佳掩模成为一个受限制掩模大小的约束的最小化问题。目标是在将相关组件保持在一定大小的同时，生成最大限度地减少失真的模糊处理。对这种方法的改进需要从几何的角度来看待这一过程，其中这种混淆掩码可以被解释为在模型映射下稳定的子空间。这意味着在给定噪声向量遵循微扰器分布的情况下，该遮罩的模型下的失真较低。创新之处在于为时间序列选择正确的表示，以便在新坐标下使所选遮罩的失真最小化。除了数据表示，解释的意义在很大程度上取决于扰动分布的性质。我们的目标是选择随机向量的这种分布，以便模糊数据位于模型训练所在的自然数据流形上，因为该流形之外可能存在模型的未开发区域，这可能会导致错误的解释。为了实现这一点，我们使用生成性对抗网络，其中我们训练生成器的方式，使生成的数据根据鉴别器与训练集中的数据无法区分。这样，生成的数据位于期望的数据流形上，并且可以基于生成器的潜变量来选择扰动分布。因此，通过使混淆不完全随机，而是使它们类似于真实世界的时间序列，这有望通过减少损坏的解释来实现更好的解释性。该项目属于EPSRC人工智能技术研究领域。

Given a Neural Network, and without prior knowledge of how it was trained, we want to analyze how it reaches decisions. In this sense, explainability is about opening the black box and determining what are the key features of the input data that contribute to a certain output.Explainability of Neural Networks is important in settings where a user is involved, such as an application setting where users apply for a position or get a medical evaluation and that is processed by a network based on deep learning techniques. The goal is to extract from the algorithm why this particular application decision or medical diagnosis was reached, and thus improving transparency and trustworthiness of the model. Explainable AI can also help scientists get better feedback to make improvements as well as get insights about their data.Explainability has most often been associated with images, due to the intuitive correspondence of features to pixels. The aim is to extend explainability to other modalities, such as time series, which are ever present in Deep Learning techniques and associated with many fields, including medical and financial. This expansion requires a generalization on the definition of feature relevance, and this is done through Rate-Distortion theory. In order to assign relevance to more complex derivatives of features, we use appropriate decompositions of time series, such as the Fourier transform. Then a small relevant component is chosen, and the rest of the data is obfuscated using a random noise vector that follows a perturbation distribution. After defining a distortion metric appropriate for classification, we can evaluate the expected distortion of the obfuscation mask over the random noise vectors. Finding the best mask therefore becomes a minimization problem subject to limiting the size of the mask. The goal is to generate an obfuscation that minimizes distortion while keeping the relevant component under a certain size. This optimal relevant component is then the explanation for the decision.Improving upon this method requires viewing this procedure from a geometric viewpoint, where this obfuscation mask can be interpreted as a subspace that is stable under the model's mapping. This means that distortion is low under the model for that mask given the noise vector follows the perturbator distribution. The innovation comes in selecting the right representation for time series so that distortion is minimized for selected masks under the new coordinates. In addition to the data representation, the meaning of an explanation greatly depends on the nature of the perturbation distribution. The goal is to select this distribution of random vectors so that the obfuscated data lies on the natural data manifold that the model was trained on, because outside this manifold may lie undeveloped regions of the model, which can result in corrupted explanations. In order to achieve this, we use a generative adversarial network, where we train a generator in such a way that the generated data is indistinguishable from the data in the training set according to a discriminator. This way, the generated data lies on the desired data manifold, and the choice of perturbator distribution can be made based on the latent variables of the generator. Therefore, by making obfuscations not entirely random, but instead have them resemble real-world time series, which is expected to achieve improved explainability by reducing corrupted explanations.This project falls within the EPSRC Artificial Intelligence Technologies research area.