Improving the Transparency of Machine Learning and Intelligent Systems for Autonomy

机器智能的许多最新发展都集中在训练和数据集的质量上，这些数据集被输入到学习算法中。智能系统的缺陷通常可以归因于训练数据的缺陷，然而，提供一个详尽的训练数据集来保证极端操作场景下的鲁棒性是不现实的。另一方面，智能算法的内部工作，包括学习算法，往往被视为一个“黑盒子”。这种方法——再加上对训练数据质量的依赖——对整个人工智能系统的验证和可解释性构成了巨大的挑战。该项目将开发用于检查具有代表性的人工智能算法的内部工作方法和方法，以增加鲁棒性。该项目旨在创建可解释性和透明度的工具和方法，重点是运输系统的应用，例如支持安全保障和自主。该项目还旨在开发工具和模型，以得出人工智能系统“学到了什么以及为什么学”。该项目的目标是(i)了解哪些已建立的和最新的机器学习（ML）技术与当前和预期的智能交通系统最相关，（ii）了解这些ML技术，应用程序和现有透明度方法（如代理模型和显著性地图）之间的映射关系，并确定主要挑战，以及（iii）开发新的透明度方法来解决其中的一些挑战。该研究的主要新颖之处在于了解透明度如何影响智能交通应用的机器学习应用，以及创建解决现有技术局限性的方法。该项目是与TRL有限公司合作的，TRL有限公司除了部分资助该项目外，还将贡献员工时间来监督学生，并在相应的支持下访问专有数据。

Many recent developments in machine intelligence focus on the quality of training and datasets that are fed into the learning algorithms. The shortfalls of an intelligent system can often be attributed to flaws in the training data, however it is not practical to provide an exhaustive training dataset to guarantee robust behaviour in extreme operating scenarios. On the other hand, the inner workings of intelligent algorithms, including learning algorithms, are often treated as a "black box". This approach - combined with reliance on the quality of training data - poses a huge challenge for the validation and interpretability of AI systems as a whole. This project will develop methods for inspecting the inner workings of representative AI algorithms and methodologies for increasing robustness.This project aims to create tools and methods for interpretability and transparency, with a focus on applications for transport systems, e.g., to support safety assurance and autonomy. The project also aims to develop tools and models to draw out "what has been learnt and why" by an AI system. The objectives of the project are (i) to understand which of the established and recent machine learning (ML) techniques are most relevant to current and anticipated intelligent transport systems, (ii) understand the mapping between these ML techniques, the applications, and existing approaches to transparency such as proxy models and salience maps, and identify the primary challenges, and (iii) develop new methods for transparency to address some of these challenges. The main novelty of the research comes from understanding how transparency impacts on ML applications of intelligent transport applications, and in the creation of methods to address some of the limitations of existing techniques.This project is in collaboration with TRL Limited who in addition to partly funding the project will contribute staff time to supervise the student and access to proprietary data with corresponding support.