What kind of human control is necessary for AI applications within national security and defence? Digital ethics and security studies

支持人工智能的系统在大小、硬件和自主程度上各不相同。自主水平通常根据预期的“有意义的人类控制”（MHC）进行分类，这是一个反映人类需要干预系统与现实世界互动程度的指标。虽然在民用领域的使用环境和场景可能相对可预测，但在安全和国防领域开发和部署支持人工智能的系统会产生几个问题。人工智能支持的系统受到一系列技术限制，包括缺乏透明度、可解释性有限、易受对抗性攻击，从而产生一系列道德和法律问题——特别是在国际人道法中。在对安全和国防领域人类控制人工智能系统的文献进行初步审查后，主要集中在致命自主武器系统（LAWS）上，很明显MHC的概念主导了对话，现有的方法倾向于一种通用的解决方案来确保MHC。为了避免对MHC的“过度限制”和“过度允许”，本研究计划寻求建立模块化控制框架的基础，该框架将：以伦理和法律原则提供的“统一依据”为基础。例如，国际人道主义法的原则，例如军事必要性、相称性、区分原则和预防原则，可以作为一个良好的起点。2. 制定一套有助于在不同任务和环境中实施这些原则的规则。这将包括评估不同环境对自主性和所需MHC水平的影响的规则。3. 提出了对这些规则进行评估、验证和验证的战争模拟模型。这也将允许通过发现某些上下文之间的最终不兼容性而出现新规则。在国际自主武器管制小组工作的基础上，这项研究将从三个层面分析MHC干预：设计控制；使用中的控制；培训。在这个框架中，虽然每个层次的干预将分别处理，但它们都将相互支持。以这种方式分解AHC将使我能够识别与上下文无关的控制组件，并根据部署的上下文建立修改控制组件的规则。然后，该框架将建立一套“如果-那么”规则，这些规则将改变赋予人工智能系统的自主程度，并适应MHC组件。

Artificial intelligence-enabled systems can vary in size, hardware and level of autonomy. The level of autonomy is typically classified according to the expected 'meaningful human control' (MHC) which is a metric reflecting the extent to which humans are required to intervene in a system's interactions with the real world. While the environment and scenarios of use may be relatively predictable in the civil domain, several issues arise from the development and deployment of AI-enabled systems in the security and defence domain. AI-enabled systems suffer from a range of technical limitations, ranging from lack of transparency, limited explicability, and susceptivity to adversarial attacks that create a range of ethical and legal issues -especially in International Humanitarian Law. After an initial review of the literature on Human Control of AI-enabled systems in the security and defence domain, which is heavily focused on Lethal Autonomous Weapons Systems (LAWS), it is clear that the MHC notion dominates the conversations and that existing approaches favour a one-size-fits-all solution to ensuring MHC. To avoid both "overly restrictive" and "overly permissive" approaches to MHC this research proposal seeks to establish the groundwork for a modular control framework that would: 1. Base itself on "unifying grounds" provided by ethical and legal principles. For example, the tenets of International Humanitarian Law, such as military necessity, proportionality, principles of distinction and precaution, serve as a good starting point. 2. Formulate a set of rules that would facilitate the operationalisation of such principles across different tasks and environments. This would include rules to assess the effect of various contexts vis-a-vis the level of autonomy and MHC required. 3. Present a war gaming model for the evaluation, verification and validation of such rules. This would also allow new rules to emerge through the discovery of eventual incompatibilities across certain contexts. Building on the work of the International Panel on Regulation of Autonomous Weapons, this research would analyse MHC across three levels of intervention: Control by design; Control in use; Training. In this framework, while each level of intervention would be addressed separately, they would all support each other. Breaking down AHC in such a way would allow me to identify context-agnostics control components and establish rules for the modification of control components according to the context of deployment. The framework would then establish a set of "if-then" rules that would change the level of autonomy given to AI-enabled systems and adapt MHC components.