Assuring Responsibility for Trustworthy Autonomous Systems

确保值得信赖的自治系统的责任

• 批准号: EP/W011239/1
• 负责人: Ibrahim Habli
• 金额: $ 89.65万
• 依托单位: University of York
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FW011239%2F1
• 关键词: Assuring; Responsibility; Trustworthy; Autonomous; Systems

Autonomous systems, such as medical systems, autonomous aerial and road vehicles, and manufacturing and agricultural robots, promise to extend and expand human capacities. But their benefits will only be harnessed if people have trust in the human processes around their design, development, and deployment. Enabling designers, engineers, developers, regulators, operators, and users to trace and allocate responsibility for the decisions, actions, failures, and outcomes of autonomous systems will be essential to this ecosystem of trust. If a self-driving car takes an action that affects you, you will want to know who is responsible for it and what are the channels for redress. If you are a doctor using an autonomous system in a clinical setting, you will want to understand the distribution of accountability between you, the healthcare organisation, and the developers of the system. Designers and engineers need clarity about what responsibilities fall on them, and when these transfer to other agents in the decision-making network. Manufacturers need to understand what they would be legally liable for. Mechanisms to achieve this transparency will not only provide all stakeholders with reassurance, they will also increase clarity, confidence, and competence amongst decision-makers. The research project is an interdisciplinary programme of work - drawing on the disciplines of engineering, law, and philosophy - that culminates in a methodology to achieve precisely that tracing and allocation of responsibility. By 'tracing responsibility' we mean the process of tracking the autonomous system's decisions or outcomes back to the decisions of designers, engineers, or operators, and understanding what led to the outcome. By 'allocating responsibility' we mean both allocating role responsibilities to different agents across the life-cycle and working out in advance who would be legally liable and morally responsible for different system decisions and outcomes once they have occurred. This methodology will facilitate responsibility-by-design and responsibility-through-lifecycle. In practice, the tracing and allocation of responsibility for the decisions and outcomes of AS is very complex. The complexity of the systems and the constant movement and unpredictability of their operational environments makes individual causal contributions difficult to distinguish. When this is combined with the fact that we delegate tasks to systems that require ethical judgement and lawful behaviour in human beings, it also gives rise to potential moral and legal responsibility gaps. The more complex and autonomous the system is, the more significant the role that assurance will play in tracing and allocating responsibility, especially in contexts that are technically and organisationally complex. The research project tackles these challenges head on. First, we clarify the fundamental concepts of responsibility, the different kinds of responsibility in play, the different agents involved, and where 'responsibility gaps' arise and how they can be addressed. Second, we build on techniques used in the technical assurance of high-risk systems to reason about responsibility in the context of uncertainty and dynamism, and therefore unpredictable socio-technical environments. Together, these strands of work provide the basis for a methodology for responsibility-by-design and responsibility-through-lifecycle that can be used in practice by a wide range of stakeholders. Assurance of responsibility will be achieved that not only identifies which agents are responsible for which outcomes and in what way throughout the lifecycle, and explains how this identification is achieved, but also establishes why this tracing and allocation of responsibility is well-justified and complete.

自主系统，如医疗系统、自主航空和公路车辆以及制造和农业机器人，有望扩展和扩大人类的能力。但是，只有当人们信任围绕其设计、开发和部署的人类过程时，它们的好处才能得到利用。让设计师、工程师、开发人员、监管机构、运营商和用户能够追踪和分配自主系统的决策、行动、故障和结果的责任，这对这个信任生态系统至关重要。如果一辆自动驾驶汽车采取了影响你的行动，你会想知道谁对此负责，以及有什么补救渠道。如果您是一名在临床环境中使用自主系统的医生，您将希望了解您、医疗保健组织和系统开发人员之间的责任分配。设计师和工程师需要明确他们的责任，以及这些责任何时转移到决策网络中的其他代理人。制造商需要了解他们将承担什么法律责任。实现这一透明度的机制不仅将为所有利益攸关方提供保证，还将提高决策者的透明度、信心和能力。该研究项目是一个跨学科的工作方案-借鉴工程，法律和哲学的学科-最终在一种方法，以实现准确的跟踪和分配的责任。所谓“追溯责任”，我们指的是将自主系统的决策或结果追溯到设计师、工程师或操作员的决策，并了解导致结果的原因的过程。通过“分配责任”，我们意味着在整个生命周期中将角色责任分配给不同的代理人，并提前确定谁将对不同的系统决策和结果承担法律责任和道德责任。这种方法将促进设计责任和生命周期责任。在实践中，AS的决策和结果的责任追踪和分配非常复杂。这些系统的复杂性及其运作环境的不断变化和不可预测性，使得难以区分个别的因果作用。当这一点与我们将任务交给需要人类道德判断和合法行为的系统这一事实结合在一起时，它还产生了潜在的道德和法律的责任差距。系统越复杂和自治，保证在追踪和分配责任方面的作用就越重要，特别是在技术和组织复杂的情况下。该研究项目正面应对这些挑战。首先，我们澄清了责任的基本概念，不同类型的责任，涉及的不同代理人，以及“责任差距”出现的地方以及如何解决这些问题。第二，我们建立在高风险系统的技术保证中使用的技术，在不确定性和动态性的背景下，因此不可预测的社会技术环境的原因有关的责任。总之，这些工作链为设计责任和生命周期责任的方法提供了基础，可供广泛的利益相关者在实践中使用。将实现责任保证，不仅确定哪些代理人在整个生命周期中以何种方式对哪些结果负责，并解释如何实现这种识别，而且还确定为什么这种责任追踪和分配是合理和完整的。

项目成果

Distinguishing two features of accountability for AI technologies

区分人工智能技术问责制的两个特征

• DOI: 10.1038/s42256-022-00533-0
• 发表时间: 2022
• 期刊: Nature Machine Intelligence
• 影响因子: 23.8
• 作者: Porter Z

Computer Safety, Reliability, and Security - 42nd International Conference, SAFECOMP 2023, Toulouse, France, September 20-22, 2023, Proceedings

计算机安全、可靠性和安保 - 第 42 届国际会议，SAFECOMP 2023，法国图卢兹，2023 年 9 月 20-22 日，会议记录

• DOI: 10.1007/978-3-031-40923-3_16
• 发表时间: 2023
• 作者: Ryan Conmy P

A principles-based ethics assurance argument pattern for AI and autonomous systems

人工智能和自主系统基于原则的道德保证论证模式

• DOI: 10.1007/s43681-023-00297-2
• 发表时间: 2023
• 期刊: AI and Ethics
• 作者: Porter Z

Safety engineering, role responsibility and lessons from the Uber ATG Tempe Accident

安全工程、角色责任以及 Uber ATG 坦佩事故的教训

• DOI: 10.1145/3597512.3599718
• 发表时间: 2023
• 作者: Ryan Conmy P

Predicting Progression of Type 2 Diabetes using Primary Care Data with the Help of Machine Learning

借助机器学习，使用初级保健数据预测 2 型糖尿病的进展

• 发表时间: 2023
• 作者: Berk OZTURK