Collaborative Research: RI: Small: Advancing Theory and Practice of Trustworthy Machine Learning via Bi-Level Optimization

合作研究：RI：小型：通过双层优化推进可信机器学习的理论和实践

• 批准号: 2207053
• 负责人: Shiyu Chang
• 金额: $ 30万
• 依托单位: University of California-Santa Barbara
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2207053&HistoricalAwards=false
• 关键词: Collaborative; Research; RI; Small; Advancing

Deep learning (DL) has achieved remarkable success owing to its superior prediction ability, with a wide range of applications in computer vision and natural language processing. Yet, one of its critical shortcomings is the lack of trustworthiness. That is, they are often overcooked during training such that (1) the learned model is highly vulnerable to small input perturbations at the testing time (namely, lack of robustness); And (2) biased artifacts embedded in the training data can be memorized and then passed on to the decision making process (namely, lack of fairness). To address these issues, this project attempts to develop a new family of trustworthy learning algorithms with algorithmic generality, theoretical soundness, and scalability to large-scale datasets and models. The outcome of this project could create a new optimization foundation of trustworthy DL that can not only unit robustness and fairness into one coherent learning paradigm but also expand the applicability of DL to a series of high-stakes applications such as autonomous driving and cybersecurity. Interdisciplinary training in computer science, applied mathematics, and engineering will be provided to all-level students, especially for students from underrepresented groups. The main technical aim of this project is to advance the theoretical understanding and practical implementations of trustworthy DL through the lens of bi-level optimization (BLO), namely, hierarchical learning involving two nested optimization tasks. The research plan consists of three thrusts. The first thrust develops a new BLO-oriented robust learning framework including defenses against adversarial instances and distribution shifts. The developed technique is also applied to building a full-stack (from train time to test time) robustness evaluation pipeline. The second thrust expands the first one and develops BLO algorithms to co-improve robustness and fairness in two practical scenarios, learning without sensitive attribute annotation, and learning with scarce training data and model information. The third thrust focuses on developing scalable and theoretically-grounded computational methods for BLO so as to achieve a high-accuracy, high-resilience, and high-throughput trustworthy learning paradigm. The project will result in the dissemination of shared toolbox and benchmarks to the broader optimization and machine learning communities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习因其优越的预测能力而取得了显著的成功，在计算机视觉和自然语言处理中有着广泛的应用。然而，它的一个关键缺点是缺乏可信度。也就是说，它们经常在训练过程中被过度煮熟，从而(1)学习的模型在测试时非常容易受到小的输入扰动(即，缺乏稳健性)；以及(2)嵌入在训练数据中的有偏差的伪像可以被记忆，然后传递到决策过程(即，缺乏公平性)。为了解决这些问题，本项目试图开发一系列新的可信学习算法，具有算法通用性、理论可靠性和对大规模数据集和模型的可伸缩性。该项目的成果可以创建一个新的可信DL的优化基础，不仅可以将稳健性和公平性统一到一个连贯的学习范式中，而且还可以将DL的适用性扩展到一系列高风险的应用，如自动驾驶和网络安全。将向所有层次的学生提供计算机科学、应用数学和工程方面的跨学科培训，特别是针对代表性不足群体的学生。该项目的主要技术目标是通过双层优化(BLO)的镜头，即涉及两个嵌套优化任务的分层学习，促进对可信DL的理论理解和实际实现。这项研究计划包括三个推进。第一个推力开发了一个新的面向BLO的稳健学习框架，包括对敌对实例和分布转移的防御。所开发的技术还应用于构建从训练时间到测试时间的全栈健壮性评估流水线。第二个推力扩展了第一个推力，并开发了BLO算法，以在两个实际场景中共同提高稳健性和公平性，即无需敏感属性注释的学习和利用稀缺的训练数据和模型信息的学习。第三个重点是为BLO开发可扩展的、有理论基础的计算方法，以实现高精度、高弹性和高通量的可信学习范式。该项目将导致向更广泛的优化和机器学习社区传播共享工具箱和基准。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。