UKRI Trustworthy Autonomous Systems Node in Functionality

“自主系统”是具有某种形式的决策能力的机器，这使得它们能够独立于人类控制器而行动。这种技术已经无处不在，从汽车的牵引力控制系统，到移动的手机和电脑（Siri，Alexa，Cortana）中的有用助手。其中一些系统比其他系统具有更多的自主性，这意味着一些系统非常可预测，并且只会以最初设置的方式做出反应，而其他系统则具有更多的自由度，可以以超出其初始设置的方式进行学习和反应。这可以使它们更有用，但也更不可预测。一些自治系统有可能改变它们的行为，我们称之为“进化功能”。这意味着，一个被设计成以某种方式完成某种任务的系统，可能会随着时间的推移而“进化”，要么以不同的方式完成相同的任务，要么完成不同的任务。所有这些都没有人类控制器告诉它该做什么。这些类型的系统正在开发，因为它们可能非常有用，具有广泛的可能应用，从最小的停机时间制造到紧急响应和机器人手术。功能演进的能力为自主系统提供了从在可预测的情况下执行定义明确的任务到在不断变化的现实环境中执行复杂任务的潜力。然而，功能演进的系统会导致对安全、责任和信任的合理担忧。我们学会信任技术，因为它是可靠的，当一项技术不可靠时，我们放弃它，因为它不能正常工作。但要学会信任功能不断变化的技术可能很困难。我们还可能会问一些重要的问题，比如如何以适当的方式监控、测试和监管功能演进的安全性。例如，仅仅因为一个能够适应处理不同形状物体的机器人通过了仓库中的安全测试，并不意味着如果它被用来在手术环境中执行类似的任务，它就一定是安全的。同样不清楚的是，如果有人的话，谁应该对功能进化的结果负责--无论是积极的还是消极的。这项研究旨在探索和解决这些问题，通过询问我们如何能够或应该信任具有不断发展的功能的自治系统。我们的方法是使用三种不断发展的技术-群体系统，软机器人和无人驾驶飞行器-它们以根本不同的方式运行，使我们的研究结果能够用于广泛的不同应用领域。我们将在真实的时间研究这些系统，探索这些系统是如何开发的，以及如何将功能内置到设计过程中，以增加可信度，称为可信设计。这将支持自主系统的开发，使其具有适应、发展和改进的能力，但要确保这些系统的开发方法确保它们是安全、可靠和值得信赖的。

'Autonomous systems' are machines with some form of decision-making ability, which allows them to act independently from a human controller. This kind of technology is already all around us, from traction control systems in cars, to the helpful assistant in mobile phones and computers (Siri, Alexa, Cortana). Some of these systems have more autonomy than others, meaning that some are very predictable and will only react in the way they are initially set up, whereas others have more freedom and can learn and react in ways that go beyond their initial setup. This can make them more useful, but also less predictable.Some autonomous systems have the potential to change what they do, and we call this 'evolving functionality'. This means that a system designed to do a certain task in a certain way, may 'evolve' over time to either do the same task a different way, or to do a different task. All without a human controller telling it what to do. These kinds of systems are being developed because they are potentially very useful, with a wide range of possible applications ranging from minimal down-time manufacturing through to emergency response and robotic surgery. The ability to evolve in functionality offers the potential for autonomous systems to move from conducting well defined tasks in predictable situations, to undertaking complex tasks in changing real-world environments.However, systems that can evolve in function lead to legitimate concerns about safety, responsibility and trust. We learn to trust technology because it is reliable, and when a technology is not reliable, we discard it because it cannot be trusted to function properly. But it may be difficult to learn to trust technology whose function is changing. We might also ask important questions about how functional evolutions are monitored, tested and regulated for safety in appropriate ways. For example, just because a robot with the ability to adapt to handle different shaped objects passes safety testing in a warehouse does not mean that it will necessarily be safe if it is used to do a similar task in a surgical setting. It is also unclear who, if anyone, bears the responsibility for the outcome of functional evolution - whether positive or negative. This research seeks to explore and address these issues, by asking how we can, or should, place trust in autonomous systems with evolving functionality. Our approach is to use three evolving technologies - swarm systems, soft robotics and unmanned air vehicles - which operate in fundamentally different ways, to allow our findings to be used across a wide range of different application areas. We will study these systems in real time to explore both how these systems are developed and how features can be built into the design process to increase trustworthiness, termed Design-for-Trustworthiness. This will support the development of autonomous systems with the ability to adapt, evolve and improve, but with the reassurance that these systems have been developed with methods that ensure they are safe, reliable, and trustworthy.