Collaborative Research: SaTC: CORE: Small: Detecting and Localizing Non-Functional Vulnerabilities in Machine Learning Libraries

该项目旨在提高机器学习（ML）软件的安全性和弹性。机器学习已被部署在许多关键领域，如药物发现、财务规划、自动驾驶和恶意软件检测。这使得基于ML的软件解决方案即使在受到恶意攻击时也能正常运行至关重要，这导致了一系列专注于功能漏洞的研究，这些攻击试图使ML系统产生错误的结果。然而，研究较少的是其他类型的漏洞，这些漏洞不会攻击核心预测功能，但仍然构成安全风险。这些“非功能性”漏洞包括拒绝服务攻击，这种攻击试图通过使系统过载而使系统无法使用;和侧通道攻击，分析响应时间等特征，推断出有关模型或数据的敏感信息。该项目将开发检测和纠正这类非功能性漏洞的方法，并使这些方法广泛使用，并传播教育材料，以帮助安全研究人员和ML软件开发人员更加了解这些风险。尽管在核心ML库（如NumPy和TensorFlow）中报告了越来越多的拒绝服务（DoS）和侧通道（SC）漏洞，但由于多个技术挑战，尚未探索识别和调试它们的系统方法：i）非功能性行为未显式编码在ML代码的语法或语义中; ii）现有的故障定位方法通常无法建立因果关系;以及iii）ML应用程序在很大程度上缺乏自动DoS/SC缓解。该项目将开发一种新的方法，将进化算法与基于梯度的指导相结合，以检测DoS并量化SC漏洞的强度。对于调试，该项目探索因果导向的统计方法，以定位根本原因并指导最佳缓解政策。该项目团队将通过专题课程、本科生研究体验和K-12学生夏令营，共同努力提高女性、西班牙裔和其他代表性不足的社区的参与度。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to improve security and resilience of machine learning (ML) software. Machine learning has been deployed in many critical domains such as drug discovery, financial planning, autonomous driving, and malware detection. This makes it crucial for ML-based software solutions to function properly even when attacked by malicious actors, leading to a line of research focused on functional vulnerabilities, attacks that attempt to make ML systems produce incorrect results. Less studied, however, are other kinds of vulnerabilities that don’t attack the core prediction functionality but still pose security risks. These “non-functional” vulnerabilities include denial of service attacks, which attempt to render the system unusable through overloading it; and side-channel attacks, which analyze features like response time to infer sensitive information about the models or data they are trained on. This project will develop methods for detecting and correcting these kinds of non-functional vulnerabilities and make those methods widely available, as well as disseminate educational materials to help security researchers and ML software developers be more aware of these risks. Despite a growing number of reported denial-of-service (DoS) and side channel (SC) vulnerabilities in core ML libraries such as NumPy and TensorFlow, a systematic approach to identifying and debugging them has not been explored due to multiple technical challenges: i) non-functional behaviors are not explicitly encoded in the syntax or semantics of ML code; ii) existing fault localization methods often fail to establish causal relationships; and iii) automatic DoS/SC mitigation is largely lacking for ML applications. This project will develop a novel methodology that combines evolutionary algorithms with a gradient-based guidance to detect DoS and quantify the strengths of SC vulnerabilities. For debugging, the project explores causally guided statistical methods to localize the root causes and guide an optimal mitigation policy. The project team will make a concerted effort to increase participation of women, Hispanic, and other underrepresented communities via special topic courses, research experiences for undergraduates, and summer camps for K-12 students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.