Supervisory Agent for Safety Improvements in AI Systems

人工智能系统安全改进的监管代理

• 批准号: 2102490
• 金额: --
• 依托单位: University of Nottingham
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2018
• 资助国家: 英国
• 起止时间: 2018 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2102490
• 关键词: Supervisory; Agent; Safety; Improvements; AI

This research will investigate the feasibility of introducing an independent Supervisory Agent to monitor AI based agent(s) operating within an environment. The primary focus is agent(s) whose behaviour has the potential to change at run-time (for example, via techniques such as generative planning or machine learning) thus introducing unknown actions. The Supervisory Agent will monitor the intended actions of the agent(s), and prevent the execution of any harmful or unsafe actions. This may involve substituting a safe action for an unsafe action that has been suppressed.

本研究将探讨引入一个独立的监督代理来监控在环境中运行的基于AI的代理的可行性。主要关注的是代理，其行为有可能在运行时发生变化（例如，通过生成规划或机器学习等技术），从而引入未知的动作。监督代理人将监控代理人的预期行为，并防止执行任何有害或不安全的行为。这可能涉及用安全操作替换已被抑制的不安全操作。