CAREER: Trustworthy Machine Learning from Untrusted Models

今天的许多基于机器学习(ML)的系统不是从头开始构建的，而是由一系列预先训练的第三方模型“组合”而成的。与其他形式的软件重用并行，重用模型既可以加快并简化基于ML的系统的开发。然而，缺乏对第三方ML模型的标准化、监管和验证引发了安全方面的担忧。特别是，ML模型容易受到对抗性攻击，在这种攻击中，第三方攻击者或模型提供者本身可能会嵌入由预先指定的输入触发的隐藏行为。该项目旨在了解重用第三方模型作为ML系统的构建块所带来的安全威胁，并开发工具帮助开发人员在ML系统的整个生命周期中缓解此类威胁。该项目的成果将在短期内提高从自动驾驶汽车到身份验证等应用程序中的ML安全性，同时从长远来看，促进构建和运营ML系统的更有原则性的做法。重用第三方模型带来的一种主要类型的威胁是模型重用攻击，在这种攻击中，恶意制作的模型(“对抗性模型”)迫使宿主ML系统以高度可预测的方式在目标输入(“触发器”)上发生故障。该项目开发了严格而实用的方法来主动检测和补救此类后门漏洞。首先，对模型重用攻击的必要条件和不变模式进行了实证和分析。其次，利用这些洞察力，它将开发一系列缓解工具，检测潜在的后门，精确定位触发因素，并提供机制来加强对抗这些攻击的模型。第三，建立对抗性模型和对抗性投入的统一理论，加深对对抗性最大似然的更一般的理解。最后，它将以原型试验台的形式实施所有提出的技术和系统设计，为研究一系列攻防技术提供独特的研究设施。在这个项目中开发的新理论和技术将被整合到本科和研究生教育中，并用于提高公众对ML安全重要性的认识。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Many of today's machine learning (ML)-based systems are not built from scratch, but are "composed" from an array of pre-trained, third-party models. Paralleling other forms of software reuse, reusing models can both speed up and simplify the development of ML-based systems. However, a lack of standardization, regulation, and verification of third-party ML models raises security concerns. In particular, ML models are subject to adversarial attacks in which third-party attackers or model providers themselves might embed hidden behaviors that are triggered by pre-specified inputs. This project aims at understanding the security threats incurred by reusing third-party models as building blocks of ML systems and developing tools to help developers mitigate such threats throughout the lifecycle of ML systems. Outcomes from the project will improve ML security in applications from self-driving cars to authentication in the short term while promoting more principled practices of building and operating ML systems in the long run.One major type of threat incurred by reusing third-party models is model reuse attacks, in which maliciously crafted models ("adversarial models") force host ML systems to malfunction on targeted inputs ("triggers") in a highly predictable manner. This project develops rigorous yet practical methods to proactively detect and remediate such backdoor vulnerabilities. First, it will empirically and analytically investigate the necessary conditions and invariant patterns of model reuse attacks. Second, leveraging these insights, it will develop a chain of mitigation tools that detect potential backdoors, pinpoint triggers, and provide mechanisms to fortify adversarial models against these attacks. Third, it will establish a unified theory of adversarial models and adversarial inputs to deepen more general understanding of adversarial ML. Finally, it will implement all the proposed techniques and system designs in the form of a prototype testbed, which provides a unique research facility for investigating a range of attack and defense techniques. New theories and techniques developed in this project will be integrated into undergraduate and graduate education and used to raise public awareness of the importance of ML security.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.