UKRI Trustworthy Autonomous Systems Node in Verifiability

自主系统有望改善我们的生活;无人驾驶火车和机器人清洁工就是已经存在的自主系统的例子，它们在封闭的环境中工作得很好。现在是我们努力确保开发人员能够为动态环境设计值得信赖的自主系统并提供其可信度证据的时候了。由于自主系统的复杂性，通常涉及AI组件，低级硬件控制以及与人类和不确定环境的复杂交互，任何性质的证据都需要各种学科的努力。为了应对这一挑战，我们召集了人工智能、机器人、人机交互、系统和软件工程以及测试方面的专家联盟。我们将共同建立验证自治系统属性的基础和技术，为设计提供信息，提供关键属性的证据，并指导部署后的监控。目前，可验证性受到以下几个问题的阻碍：难以理解的是，专注于系统各个方面的技术如何提供证据（例如，控制工程，人工智能或人机交互）组成为整个系统提供证据;在其学科中使用不同语言和实践的利益相关者之间的沟通困难;在处理人工智能、控制和硬件设计、关键系统软件等先进概念方面的困难;因此，自主系统的开发往往使用先进的工程技术，但验证方法却过时了。我们提出了一个创造性的工作计划，将使当前的技术和实践状态发生根本性的变化。我们将定义一个数学框架，使人们能够对自治验证中涉及的各种实践和概念有一个共同的理解。我们的框架将提供任何工程工作所需的数学基础，以适应各种学科所使用的符号。有了这个共识，我们将证明语言之间的翻译、用不同语言定义的人工制品（工程模型、测试、模拟等）的组合以及从组件验证中得出的系统级推论是正确的。有了这样一个丰富的基础和丰富的结果，我们将改变实践的状态。目前，开发人员从头开始构建系统，或者在没有任何操作条件证据的情况下重用组件。由此产生的系统部署在约束条件下（例如，降低速度或封闭环境）或提供部署，风险由用户自己承担。相反，我们设想的是，未来可以提供一个经过验证的自主系统和组件的存储库。在这样的未来，用户在存储库中不仅可以找到系统实现，还可以找到其运行条件和预期行为的证据（工程模型、数学结果、测试等）。当开发人员签入产品时，商店将要求所有这些以易于理解的语言描述的事实，并将自动验证可信度的证据。开发人员也可以为其他开发人员签入组件;同样，他们也会得到所需的证据，以确保他们对自己的使用有信心。在这个变化了的世界里，用户购买的应用程序将明确保证他们的操作需求和配置文件。例如，用户还可以要求验证定制平台和环境的充分性。我们将与EPSRC TAS Hub和其他节点以及我们广泛的学术和工业合作伙伴合作，确保我们考虑的标记、验证技术和属性有助于我们的共同议程，将自主权带入我们的日常生活。

Autonomous systems promise to improve our lives; driverless trains and robotic cleaners are examples of autonomous systems that are already among us and work well within confined environments. It is time we work to ensure developers can design trustworthy autonomous systems for dynamic environments and provide evidence of their trustworthiness.Due to the complexity of autonomous systems, typically involving AIcomponents, low-level hardware control, and sophisticated interactions withhumans and the uncertain environment, evidence of any nature requires efforts from avariety of disciplines. To tackle this challenge, we gathered consortium ofexperts on AI, robotics, human-computer interaction, systems and softwareengineering, and testing. Together, we will establish the foundations andtechniques for verification of properties of autonomous systems to informdesigns, provide evidence of key properties, and guide monitoringafter deployment.Currently, verifiability is hampered by several issues: difficulties tounderstand how evidence provided by techniques that focus on individualaspects of a system (control engineering, AI, or humaninteraction, for example) compose to provide evidence for the system aswhole; difficulties of communication between stakeholders that use differentlanguages and practices in their disciplines; difficulties in dealing withadvanced concepts in AI, control and hardware design, software for criticalsystems; and others. As a consequence, autonomous systems are often developedusing advanced engineering techniques, but outdated approaches toverification.We propose a creative programme of work that will enable fundamental changesto the current state of the art and of practice. We will define amathematical framework that enables a common understanding of the diversepractices and concepts involved in verification of autonomy. Our frameworkwill provide the mathematical underpinning, required by any engineeringeffort, to accommodate the notations used by the various disciplines. Withthis common understanding, we will justify translations between languages,compositions of artefacts (engineering models, tests, simulations, and so on)defined in different languages, and system-level inferences fromverifications of components.With such a rich foundation and wealth of results, we willtransform the state of practice. Currently, developers build systems fromscratch, or reusing components without any evidence of theiroperational conditions. Resulting systems are deployed in constrainedconditions (reduced speed or contained environment, for example) or offeredfor deployment at the user's own risk. Instead, we envisage the futureavailability of a store of verified autonomous systems and components.In such a future, in the store, users will find not just systemimplementations, but also evidence of their operational conditions andexpected behaviour (engineering models, mathematical results, tests, and soon). When a developer checks in a product, the store will require all theseartefacts, described in well understood languages, and will automaticallyverify the evidence of trustworthiness. Developers will also be able to checkin components for other developers; equally, they will be accompanied byevidence required to permit confidence in their use.In this changed world, users will buy applications with clear guarantees oftheir operational requirements and profile. Users will also be able to askfor verification of adequacy for customised platforms and environment, forexample. Verification is no longer an issue.Working with the EPSRC TAS Hub and other nodes, and our extensive range ofacademic and industrial partners, we will collaborate to ensure that thenotations, verification techniques, and properties, that we consider,contribute to our common agenda to bring autonomy to our everyday lives.