SAIS: Secure AI assistantS

SAIS：安全人工智能助手

• 批准号: EP/T026723/1
• 负责人: Jose Such
• 金额: $ 147.21万
• 依托单位: King's College London
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2020
• 资助国家: 英国
• 起止时间: 2020 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FT026723%2F1
• 关键词: SAIS; Secure; AI; assistantS

There is an unprecedented integration of AI assistants into everyday life, from the personal AI assistants running in our smart phones and homes, to enterprise AI assistants for increased productivity at the workplace, to health AI assistants. Only in the UK, 7M users interact with AI assistants every day, and 13M on a weekly basis. A crucial issue is how secure AI assistants are, as they make extensive use of AI and learn continually. Also, AI assistants are complex systems with different AI models interacting with each other and with the various stakeholders and the wider ecosystem in which AI assistants are embedded. This ranges from adversarial settings, where malicious actors exploit vulnerabilities that arise from the use of AI models to make AI assistants behave in an insecure way, to accidental ones, where negligent actors introduce security issues or use AIS insecurely. Beyond the technical complexities, users of AI assistants are known to have mental models that are highly incomplete and they do not know how to protect themselves. SAIS (Secure AI assistantS) is a cross-disciplinary collaboration between the Departments of Informatics, Digital Humanities and The Policy Institute at King's College London, and the Department of Computing at Imperial College London, working with non-academic partners: Microsoft, Humley, Hospify, Mycroft, policy and regulation experts, and the general public, including non-technical users. SAIS will provide an understanding of attacks on AIS considering the whole AIS ecosystem, the AI models used in them, and all the stakeholders involved, particularly focusing on the feasibility and severity of potential attacks on AIS from a strategic threat and risk approach. Based on this understanding, SAIS will propose methods to specify, verify and monitor the security behaviour of AIS based on model- based AI techniques known to provide richer foundations than data-driven ones for explanations on the behaviour of AI-based systems. This will result in a multifaceted approach, including: a) novel specification and verification techniques for AIS, such as methods to verify the machine learning models used by AIS; b) novel methods to dynamically reason about the expected behaviour of AIS to be able to audit and detect any degradation or deviation from that expected behaviour based on normative systems and data provenance; iii) co-created security explanations following a techno-cultural method to increase users' literacy of AIS security in a way that users can comprehend.

人工智能助理正在前所未有地融入日常生活，从我们智能手机和家庭中运行的个人人工智能助理，到提高工作场所生产力的企业人工智能助理，再到健康人工智能助理。仅在英国，每天就有700万用户与人工智能助手互动，每周有1300万用户与人工智能助手互动。一个关键问题是人工智能助手的安全性，因为它们广泛使用人工智能并不断学习。此外，人工智能助理是复杂的系统，不同的人工智能模型相互交互，并与各种利益相关者以及嵌入人工智能助理的更广泛的生态系统交互。这包括对抗性设置，其中恶意行为者利用使用AI模型产生的漏洞使AI助手以不安全的方式行事，以及意外设置，其中疏忽的行为者引入安全问题或不安全地使用AIS。除了技术复杂性之外，人工智能助手的用户还具有高度不完整的心理模型，他们不知道如何保护自己。SAIS（安全人工智能助理）是伦敦国王学院信息学、数字人文学和政策研究所系与帝国理工学院伦敦计算系之间的跨学科合作，与非学术合作伙伴合作：微软、Humley、Hospify、Mycroft、政策和法规专家以及公众，包括非技术用户。SAIS将提供对AIS攻击的理解，考虑到整个AIS生态系统，其中使用的AI模型以及所有相关利益相关者，特别关注从战略威胁和风险方法对AIS进行潜在攻击的可行性和严重性。基于这种理解，SAIS将提出方法来指定，验证和监控基于模型的AI技术的AIS的安全行为，这些技术为解释基于AI的系统的行为提供了比数据驱动的更丰富的基础。这将产生一种多方面的方法，包括：a）用于AIS的新的规范和验证技术，例如验证AIS所使用的机器学习模型的方法; B）动态推理AIS的预期行为的新方法，以便能够基于规范系统和数据出处审计和检测与预期行为的任何降级或偏离; iii）按照技术文化方法共同创建安全解释，以用户可以理解的方式提高用户对AIS安全的认识。

项目成果

Predicting Privacy Preferences for Smart Devices as Norms

预测智能设备的隐私偏好作为规范

• 发表时间: 2023
• 作者: Serramia M

Collaborative filtering to capture AI user's preferences as norms

协同过滤捕捉人工智能用户的偏好作为规范

• DOI: 10.48550/arxiv.2308.02542
• 发表时间: 2023
• 作者: Serramia M

Measuring Alexa Skill Privacy Practices across Three Years

• DOI: 10.1145/3485447.3512289
• 发表时间: 2022-04
• 期刊: Proceedings of the ACM Web Conference 2022
• 作者: Jide S. Edu;Xavier Ferrer Aran;J. Such;Guillermo Suarez-Tangil

Exploring the security and privacy risks of chatbots in messaging services

探索消息服务中聊天机器人的安全和隐私风险

• DOI: 10.1145/3517745.3561433
• 发表时间: 2022
• 期刊: ACM Internet Measurement Conference (IMC
• 作者: Edu, Jide;Mulligan, Cliona;Pierazzi, Fabio;Polakis, Jason;Suarez-Tangil, Guillermo;Such, Jose
• 通讯作者: Such, Jose

SkillVet: Automated Traceability Analysis of Amazon Alexa Skills

• DOI: 10.1109/tdsc.2021.3129116
• 发表时间: 2021-03
• 期刊: IEEE Transactions on Dependable and Secure Computing
• 影响因子: 7.3
• 作者: Jide S. Edu;Xavier Ferrer-Aran;J. Such;Guillermo Suarez-Tangil