Collaborative Research: RI: Small: Advancing Theory and Practice of Trustworthy Machine Learning via Bi-Level Optimization

合作研究：RI：小型：通过双层优化推进可信机器学习的理论和实践

• 批准号: 2207053
• 负责人: Shiyu Chang
• 金额: $ 30万
• 依托单位: University of California-Santa Barbara
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2207053&HistoricalAwards=false
• 关键词: Collaborative; Research; RI; Small; Advancing

Deep learning (DL) has achieved remarkable success owing to its superior prediction ability, with a wide range of applications in computer vision and natural language processing. Yet, one of its critical shortcomings is the lack of trustworthiness. That is, they are often overcooked during training such that (1) the learned model is highly vulnerable to small input perturbations at the testing time (namely, lack of robustness); And (2) biased artifacts embedded in the training data can be memorized and then passed on to the decision making process (namely, lack of fairness). To address these issues, this project attempts to develop a new family of trustworthy learning algorithms with algorithmic generality, theoretical soundness, and scalability to large-scale datasets and models. The outcome of this project could create a new optimization foundation of trustworthy DL that can not only unit robustness and fairness into one coherent learning paradigm but also expand the applicability of DL to a series of high-stakes applications such as autonomous driving and cybersecurity. Interdisciplinary training in computer science, applied mathematics, and engineering will be provided to all-level students, especially for students from underrepresented groups. The main technical aim of this project is to advance the theoretical understanding and practical implementations of trustworthy DL through the lens of bi-level optimization (BLO), namely, hierarchical learning involving two nested optimization tasks. The research plan consists of three thrusts. The first thrust develops a new BLO-oriented robust learning framework including defenses against adversarial instances and distribution shifts. The developed technique is also applied to building a full-stack (from train time to test time) robustness evaluation pipeline. The second thrust expands the first one and develops BLO algorithms to co-improve robustness and fairness in two practical scenarios, learning without sensitive attribute annotation, and learning with scarce training data and model information. The third thrust focuses on developing scalable and theoretically-grounded computational methods for BLO so as to achieve a high-accuracy, high-resilience, and high-throughput trustworthy learning paradigm. The project will result in the dissemination of shared toolbox and benchmarks to the broader optimization and machine learning communities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

由于其出色的预测能力，深度学习（DL）取得了巨大的成功，并在计算机视觉和自然语言处理中进行了广泛的应用。然而，它的关键缺点之一是缺乏可信赖性。也就是说，在训练期间，它们通常会被煮熟，以便（1）在测试时间（即缺乏鲁棒性），学到的模型非常容易受到小型输入扰动的影响； （2）可以记住嵌入在培训数据中的偏见的伪像，然后传递到决策过程（即缺乏公平性）。为了解决这些问题，该项目试图开发一个具有算法通用性，理论性能以及对大规模数据集和模型的可信赖学习算法的新家族。该项目的结果可能会为可信赖的DL创造一个新的优化基础，不仅可以将稳健性和公平性单位化为一个连贯的学习范式，而且还可以将DL的适用性扩展到一系列高风险应用程序上，例如自动驾驶和网络安全。将向全层学生提供计算机科学，应用数学和工程学的跨学科培训，尤其是针对代表性不足小组的学生。该项目的主要技术目的是通过双层优化（BLO）的镜头（即层次学习涉及两个嵌套的优化任务）来推进可信赖DL的理论理解和实际实现。研究计划包括三个推力。第一个推力开发出一个新的面向闭合的稳健学习框架，包括防御对抗实例和分配变化。开发的技术还用于构建全栈（从火车时间到测试时间）鲁棒性评估管道。第二个推力扩展了第一个，并在两个实用的情况下开发了BLO算法，以共同解决稳健性和公平性，在没有敏感属性注释的情况下学习，并使用稀缺的培训数据和模型信息学习。第三个推力重点是为BLO开发可扩展和理论上的计算方法，以实现高敏锐的，高弹性和高通量值得信赖的学习范式。该项目将导致将共享工具箱和基准传播到更广泛的优化和机器学习社区。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估的评估来支持的。