Improving the Transparency of Machine Learning and Intelligent Systems for Autonomy

机器智能的许多最新发展关注于输入到学习算法的训练和数据集的质量。智能系统的不足往往可归因于训练数据中的缺陷，然而，提供详尽的训练数据集以保证在极端操作情况下的稳健行为是不切实际的。另一方面，智能算法的内部工作原理，包括学习算法，往往被视为“黑匣子”。这种方法--再加上对训练数据质量的依赖--对人工智能系统的整体验证和可解释性构成了巨大的挑战。这个项目将开发检查典型人工智能算法内部工作的方法和增强健壮性的方法。该项目旨在创建可解释性和透明度的工具和方法，重点是交通系统的应用，例如，支持安全保证和自主性。该项目还旨在开发工具和模型，以通过人工智能系统来得出“学到了什么以及为什么”。该项目的目标是(I)了解哪些已建立的和最新的机器学习(ML)技术与当前和预期的智能交通系统最相关，(Ii)了解这些机器学习技术、应用和现有透明度方法(如代理模型和显著图)之间的映射，并确定主要挑战，以及(Iii)开发新的透明度方法来应对其中一些挑战。这项研究的主要新颖性来自于了解透明度如何影响智能交通应用程序的ML应用，以及如何创建方法来解决现有技术的一些限制。该项目与TRL Limited合作，TRL Limited除了为该项目提供部分资金外，还将贡献工作人员时间来监督学生并在相应支持下访问专有数据。

Many recent developments in machine intelligence focus on the quality of training and datasets that are fed into the learning algorithms. The shortfalls of an intelligent system can often be attributed to flaws in the training data, however it is not practical to provide an exhaustive training dataset to guarantee robust behaviour in extreme operating scenarios. On the other hand, the inner workings of intelligent algorithms, including learning algorithms, are often treated as a "black box". This approach - combined with reliance on the quality of training data - poses a huge challenge for the validation and interpretability of AI systems as a whole. This project will develop methods for inspecting the inner workings of representative AI algorithms and methodologies for increasing robustness.This project aims to create tools and methods for interpretability and transparency, with a focus on applications for transport systems, e.g., to support safety assurance and autonomy. The project also aims to develop tools and models to draw out "what has been learnt and why" by an AI system. The objectives of the project are (i) to understand which of the established and recent machine learning (ML) techniques are most relevant to current and anticipated intelligent transport systems, (ii) understand the mapping between these ML techniques, the applications, and existing approaches to transparency such as proxy models and salience maps, and identify the primary challenges, and (iii) develop new methods for transparency to address some of these challenges. The main novelty of the research comes from understanding how transparency impacts on ML applications of intelligent transport applications, and in the creation of methods to address some of the limitations of existing techniques.This project is in collaboration with TRL Limited who in addition to partly funding the project will contribute staff time to supervise the student and access to proprietary data with corresponding support.