SAIS: Secure AI assistantS

人工智能助手前所未有地融入了我们的日常生活，从智能手机和家庭中的个人人工智能助手，到提高工作场所生产力的企业人工智能助手，再到健康人工智能助手。仅在英国，每天就有700万用户与人工智能助手互动，每周有1300万用户。一个关键问题是人工智能助手的安全性，因为它们广泛使用人工智能并不断学习。此外，人工智能助手是一个复杂的系统，不同的人工智能模型相互作用，与各种利益相关者和嵌入人工智能助手的更广泛的生态系统相互作用。这包括从对抗性设置（恶意行为者利用使用人工智能模型产生的漏洞使人工智能助手以不安全的方式行事）到意外设置（疏忽行为者引入安全问题或不安全地使用人工智能）。除了技术上的复杂性，人们还知道人工智能助手的用户拥有高度不完整的心智模型，他们不知道如何保护自己。SAIS（安全人工智能助手）是伦敦国王学院信息学、数字人文和政策研究所系与伦敦帝国理工学院计算机系之间的跨学科合作，与非学术合作伙伴合作：微软、Humley、Hospify、Mycroft、政策和法规专家以及包括非技术用户在内的公众。SAIS将根据整个AIS生态系统、其中使用的AI模型以及所有相关利益相关者，提供对AIS攻击的理解，特别是从战略威胁和风险的角度关注对AIS潜在攻击的可行性和严重性。基于这种理解，SAIS将提出基于基于模型的人工智能技术的方法来指定、验证和监控人工智能的安全行为，已知的基于模型的人工智能技术为解释基于人工智能的系统的行为提供比数据驱动的技术更丰富的基础。这将导致多方面的方法，包括：a) AIS的新规范和验证技术，例如验证AIS使用的机器学习模型的方法；b)动态推理AIS预期行为的新方法，以便能够根据规范系统和数据来源审核和检测任何退化或偏离预期行为的情况；iii)采用技术文化方法共同创建安全解释，以用户可以理解的方式提高用户对AIS安全的素养。

There is an unprecedented integration of AI assistants into everyday life, from the personal AI assistants running in our smart phones and homes, to enterprise AI assistants for increased productivity at the workplace, to health AI assistants. Only in the UK, 7M users interact with AI assistants every day, and 13M on a weekly basis. A crucial issue is how secure AI assistants are, as they make extensive use of AI and learn continually. Also, AI assistants are complex systems with different AI models interacting with each other and with the various stakeholders and the wider ecosystem in which AI assistants are embedded. This ranges from adversarial settings, where malicious actors exploit vulnerabilities that arise from the use of AI models to make AI assistants behave in an insecure way, to accidental ones, where negligent actors introduce security issues or use AIS insecurely. Beyond the technical complexities, users of AI assistants are known to have mental models that are highly incomplete and they do not know how to protect themselves. SAIS (Secure AI assistantS) is a cross-disciplinary collaboration between the Departments of Informatics, Digital Humanities and The Policy Institute at King's College London, and the Department of Computing at Imperial College London, working with non-academic partners: Microsoft, Humley, Hospify, Mycroft, policy and regulation experts, and the general public, including non-technical users. SAIS will provide an understanding of attacks on AIS considering the whole AIS ecosystem, the AI models used in them, and all the stakeholders involved, particularly focusing on the feasibility and severity of potential attacks on AIS from a strategic threat and risk approach. Based on this understanding, SAIS will propose methods to specify, verify and monitor the security behaviour of AIS based on model- based AI techniques known to provide richer foundations than data-driven ones for explanations on the behaviour of AI-based systems. This will result in a multifaceted approach, including: a) novel specification and verification techniques for AIS, such as methods to verify the machine learning models used by AIS; b) novel methods to dynamically reason about the expected behaviour of AIS to be able to audit and detect any degradation or deviation from that expected behaviour based on normative systems and data provenance; iii) co-created security explanations following a techno-cultural method to increase users' literacy of AIS security in a way that users can comprehend.