CAREER: Towards Provenance-Driven Understanding of Machine Learning Robustness

职业：对机器学习鲁棒性的起源驱动理解

• 批准号: 2238084
• 负责人: Birhanu Eshete
• 金额: $ 61.98万
• 依托单位: Regents of the University of Michigan - Dearborn
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-05-01 至 2028-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2238084&HistoricalAwards=false
• 关键词: CAREER; Towards; Provenance; Driven; Understanding

Machine Learning (ML) is increasingly used in socially critical applications such as self-driving cars, medicine, finance, and criminal justice. However, ML is also susceptible to adversaries who can attack both the data models are trained on and the ML models themselves. This can lead to poor behavior in the models and poor decisions in the people who use them. This project’s goal is to advance our ability to detect and respond to attacks through focusing on provenance: systematic capture of the data and training methods used in building models, along with the inference processes and decisions made after they are deployed. By capturing these data and developing methods to use the data when assessing risks, auditing models, and forensically analyzing incidents, the work will make ML systems both more robust and more accountable around attacks. These capabilities will in turn benefit organizations that develop and use ML models, along with policymakers and regulators who oversee their effects. The work is organized into three main thrusts. The first thrust focuses on systematic capture and characterization of pre-deployment (training) and post-deployment (inference) provenance, focusing on what constitutes training and inference provenance and the innate nondeterminism of ML computations. In particular, training and inference metadata, training progression, inference computation dynamics, and per-label characterization approaches will be explored. The second thrust will use these data for provenance-driven detection of training data poisoning and model evasion across a range of threat models and application domains. For poisoning detection, both similarity-based and distribution shift detection-based approaches will be pursued while for evasion detection, inference provenance will be analyzed empirically and structurally. The third thrust focuses on developing post-compromise forensics capabilities with the goal of tracing back attacks to their cause(s) and mitigating future attacks. Integrated with these three thrusts is an educational plan that includes developing new courses on ML trustworthiness for undergraduate and graduate students, robust ML-focused ethical hacking competitions for undergraduates, and K-12 summer camps on robust ML to develop and diversify the next generation of cybersecurity workers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习(ML)越来越多地应用于社会关键应用，如自动驾驶汽车、医疗、金融和刑事司法。然而，ML也容易受到对手的攻击，他们既可以攻击训练的数据模型，也可以攻击ML模型本身。这可能会导致模型中的不良行为和使用它们的人的糟糕决策。该项目的目标是通过关注来源来提高我们检测和应对攻击的能力：系统地捕获数据和在建立模型时使用的培训方法，以及在部署后做出的推理过程和决策。通过捕获这些数据并开发在评估风险、审计模型和对事件进行取证分析时使用这些数据的方法，这项工作将使ML系统在应对攻击时更加健壮和更负责任。这些能力反过来将使开发和使用ML模型的组织以及监督其影响的政策制定者和监管机构受益。这项工作被组织成三个主要推动力。第一个重点是系统地捕获和表征部署前(训练)和部署后(推理)来源，重点是什么构成训练和推理来源以及ML计算固有的不确定性。特别是，将探索训练和推理元数据、训练进度、推理计算动力学和每标签表征方法。第二个推力将使用这些数据对一系列威胁模型和应用领域中的训练数据中毒和模型规避进行来源驱动的检测。对于中毒检测，将采用基于相似性和基于分布平移检测的方法，而对于规避检测，将从经验和结构上分析推理来源。第三个重点是发展后妥协取证能力，目标是追踪攻击的原因(S)并减轻未来的攻击。与这三项努力相结合的是一项教育计划，其中包括为本科生和研究生开发关于ML可信度的新课程，为本科生开发强大的ML道德黑客竞赛，以及关于稳健ML的K-12夏令营，以培养和多样化下一代网络安全工作者。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

DeResistor: Toward Detection-Resistant Probing for Evasion of Internet Censorship

DeResistor：针对逃避互联网审查的抗检测探测

• 发表时间: 2023
• 期刊: 32nd USENIX Security Symposium
• 作者: Amich, Abderrahmen;Eshete, Birhanu;Yegneswaran, Vinod;Hoang, Nguyen Phong
• 通讯作者: Hoang, Nguyen Phong