SaTC: CORE: Small: Understanding and Mitigating the Security Risks of AutoML

SaTC：核心：小型：了解和减轻 AutoML 的安全风险

• 批准号: 2212323
• 负责人: Fenglong Ma
• 金额: $ 50万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2212323&HistoricalAwards=false
• 关键词: SaTC; CORE; Small; Understanding; Mitigating

Automated machine learning (AutoML) represents a new machine learning paradigm that automates the pipeline from raw data to deployable models, enabling a much wider range of people to use machine learning techniques. However, each stage of this pipeline is subject to malicious attacks, which can lead to inaccurate or vulnerable models. This project’s goal is to understand how both the technologies underlying AutoML and the ways it is adopted change security risks around machine learning and how possible defenses to them change when using AutoML. The success of this project will not only improve the security of AutoML but also promote more principled practices of building and operating machine learning systems in general, while contributing to knowledge in the areas of security, machine learning, and human-computer interaction. The project has three main sub-goals: accounting for the full spectrum of security risks that arise around AutoML; understanding the fundamental factors that drive such risks; and designing for machine learning practitioners without extensive expertise. To accomplish these goals, the team will (i) better understand current practices around AutoML through user studies and interviews; (ii) empirically and analytically explore the security vulnerabilities of AutoML-generated models through assessing these models on widely used datasets; (iii) analyze the results of the first two activities to develop a comprehensive accounting of underlying factors such as standardization of algorithmic choices in the technology or over-reliance on automated metrics by users; and (iv) developing new principles, methodologies, and tools to mitigate the aforementioned risks. The team will also integrate the work into a number of college courses and conduct public outreach to raise awareness of the role machine learning plays in everyday life.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

自动机器学习(AutoML)代表了一种新的机器学习范式，它自动化了从原始数据到可部署模型的管道，使更多的人能够使用机器学习技术。然而，这条管道的每个阶段都会受到恶意攻击，这可能会导致模型不准确或易受攻击。该项目的目标是了解AutoML背后的技术及其采用的方式如何改变围绕机器学习的安全风险，以及在使用AutoML时对它们的可能防御措施如何改变。该项目的成功不仅将提高AutoML的安全性，还将在总体上促进建立和操作机器学习系统的更有原则的做法，同时促进安全、机器学习和人机交互领域的知识。该项目有三个主要的子目标：考虑到围绕AutoML出现的所有安全风险；了解驱动这些风险的基本因素；以及为没有广泛专业知识的机器学习从业者进行设计。为了实现这些目标，该团队将(I)通过用户研究和访谈更好地了解AutoML的当前实践；(Ii)通过在广泛使用的数据集上评估AutoML生成的模型，以实证和分析的方式探索这些模型的安全漏洞；(Iii)分析前两项活动的结果，以全面核算潜在因素，如技术中算法选择的标准化或用户对自动化指标的过度依赖；以及(Iv)开发新的原则、方法和工具来降低上述风险。该团队还将把这项工作整合到一些大学课程中，并进行公共宣传，以提高人们对机器学习在日常生活中所起作用的认识。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

"Is your explanation stable?": A Robustness Evaluation Framework for Feature Attribution

• DOI: 10.1145/3548606.3559392
• 发表时间: 2022-09
• 期刊: Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security
• 作者: Yuyou Gan;Yuhao Mao;Xuhong Zhang;S. Ji;Yuwen Pu;Meng Han;Jianwei Yin;Ting Wang

AutoML in The Wild: Obstacles, Workarounds, and Expectations

• DOI: 10.1145/3544548.3581082
• 发表时间: 2023-02
• 期刊: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
• 作者: Yuan Sun;Qiurong Song;Xinning Gui;Fenglong Ma;Ting Wang

On the Security Risks of Knowledge Graph Reasoning

• DOI: 10.48550/arxiv.2305.02383
• 发表时间: 2023-05
• 期刊: ArXiv
• 作者: Zhaohan Xi;Tianyu Du;Changjiang Li;Ren Pang;S. Ji;Xiapu Luo;Xusheng Xiao;Fenglong Ma;Ting Wa

The Dark Side of AutoML: Towards Architectural Backdoor Search

• DOI: 10.48550/arxiv.2210.12179
• 发表时间: 2023
• 期刊: ArXiv
• 作者: Ren Pang;Changjiang Li;Zhaohan Xi;S. Ji;Ting Wang

AUTOMED: Automated Medical Risk Predictive Modeling on Electronic Health Records

• DOI: 10.1109/bibm55620.2022.9995209
• 发表时间: 2022-12
• 期刊: 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
• 作者: Suhan Cui;Jiaqi Wang;Xinning Gui;Ting Wang;Fenglong Ma