Learning for Reliable Autonomous Systems and Control

近年来，人们对使用数据和机器学习技术来开发在我们周围的物理世界中自主运行的系统越来越感兴趣。这包括从机器人系统或自动驾驶汽车到工业物联网的应用。数据驱动的方法可以补充传统的基于模型的方法，并且特别适用于先前的物理模型（例如，从第一原理出发）不可用的工程问题，以及系统运行环境随时间变化的情况。然而，尽管需要数据驱动的方法和基于模型的方法，研究界仍在试图了解如何安全可靠地使用这些方法。当依赖于有限数量的数据时，这通常是在从物理世界收集数据的场景中出现的情况，机器学习过程的输出将会不匹配，并且与模型完全已知的情况相比表现不佳。此外，最近的证据表明，当面对随机甚至恶意的扰动时，机器学习过程的输出可能会表现出意想不到的行为。该项目旨在从根本上理解鲁棒性和可靠性问题，以便开发可信任的自主系统。为了回答这些问题，该项目通过将鲁棒控制理论和鲁棒优化方法与数据驱动方法相结合，提供了一个新的视角。对于后者，特别感兴趣的是强化学习方法，因为需要设计在动态环境中运行的自主系统，并随着时间的推移从收集的数据轨迹中学习。鲁棒控制和鲁棒优化技术传统上是为研究具有不确定性的动力系统模型而开发的，而相比之下，本项目将探索它们在不确定性源于数据和机器学习技术的使用时的应用。因此，该项目的目标是双重的。首先，它将提供一个数学框架来描述强化学习方法的基本稳健性，并理解潜在扰动的最坏影响是什么。其次，它将通过重新思考数据用于设计动态自治系统的方式，开发一种方法来抵消鲁棒性的缺乏。这将通过在学习阶段考虑到鲁棒性概念的新算法来实现。这些算法将主要在计算机模拟环境中开发和测试，用于在未知和动态环境中运行的多机器人系统，并有可能在稍后阶段在概念验证平台上进行实验验证。该项目属于EPSRC人工智能和机器人主题，并且与EPSRC工程研究领域（控制工程）有着密切的联系。

Recent years have seen an increased interest in the use of data, together with machine learning techniques, to develop systems that operate autonomously in the physical world around us. This includes applications ranging from robotic systems or autonomous vehicles, to the industrial Internet-of-Things. Data-driven methods can complement traditional model-based approaches and are particularly suitable for engineering problems where prior physical models, e.g., from first principles, are unavailable, as well as in situations where the environment in which the systems operate is changing over time. However, despite the need for data-driven methodologies alongside model-based ones, the research community is still trying to understand how to employ such methods safely and reliably. When relying on limited amount of data, which is often the case in scenarios where data is collected from the physical world, the outputs of the machine learning procedure will have a mismatch and not perform equally well compared to the case where a model is completely known. Furthermore, recent evidence shows that the outputs of the machine learning procedure can behave unexpectedly when faced with perturbations which can be random or even malicious. This project aims to fundamentally understand the issues of robustness and reliability in order to enable the development of autonomous systems that can be trusted. To answer these questions, the project offers a new perspective by combining methods from robust control theory and robust optimisation with data-driven methods. For the latter, of particular interest are reinforcement learning methods, because of the need to design autonomous systems that operate in dynamic environments and learn from collected data trajectories over time. Robust control and robust optimization techniques have been traditionally developed for the study of models of dynamical systems with uncertainty, while in contrast this project will explore their use when uncertainty is stemming from the use of data and machine learning techniques. As a result, the objective of the project is twofold. First, it will provide a mathematical framework for characterising fundamentally robustness in reinforcement learning approaches and understanding what the worst-case impact of potential perturbations is. Second, it will develop a methodology for counteracting lack of robustness by rethinking the way data are used to design dynamic autonomous systems. This will be achieved with new algorithms that take into account the developed notions of robustness during the learning phase. These algorithms will be primarily developed and tested on a computer simulation environment for multi-robot systems operating in unknown and dynamic environments, with the potential for experimental validations at a later stage on a proof-of-concept platform. This project falls within the EPSRC Artificial intelligence and robotics theme, and additionally makes strong connections with the EPSRC Engineering research area (Control Engineering).