Optimization-based Implicit Deep Learning, Theory and Applications

基于优化的隐式深度学习、理论与应用

• 批准号: 2309810
• 负责人: Samy Wu Fung
• 金额: $ 29.5万
• 依托单位: Colorado School of Mines
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-15 至 2026-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2309810&HistoricalAwards=false
• 关键词: Optimization; based; Implicit; Deep; Learning

The past decade has seen remarkable success in deep learning. However, a significant challenge in today's era is to ensure interpretability and reliability in these models. In various applications, deep neural networks (DNNs) need to provide guarantees on their outputs, such as maintaining a self-driving car within its lane. On the other hand, many of these tasks can be formulated as optimization problems, where optimization algorithms offer interpretable and reliable solutions. Unfortunately, these models do not leverage data and thus fall short of state-of-the-art deep learning models. This research will address enhancing interpretability and reliability in deep learning methods and improve public safety when such learning methods are applied. In addition, the project will provide valuable educational opportunities for students involved. Participants will gain knowledge in inverse problems, optimization, and machine learning, which are transferable skills applicable in academia, government, and industry. The project aims to develop a framework that combines the interpretability and reliability of optimization algorithms with the design and training of DNNs. The primary focus is on implicit networks, a type of DNNs that determines their outputs implicitly through fixed point or optimality conditions, rather than a fixed number of computations like traditional DNNs with a set number of layers. This integration of optimization algorithms into implicit networks is referred to as implicit learning-to-optimize (L2O) networks. Implicit L2O networks have the potential to overcome the limitations of traditional DNNs, including their lack of reliability and interpretability. However, training and designing implicit L2O models present additional challenges that hinder their widespread adoption. To address these challenges, the research aims to develop a universal implicit L2O framework.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在过去的十年里，深度学习取得了巨大的成功。然而，当今时代的一个重大挑战是确保这些模型的可解释性和可靠性。在各种应用中，深度神经网络（DNN）需要为其输出提供保证，例如将自动驾驶汽车保持在车道内。另一方面，这些任务中的许多可以被公式化为优化问题，其中优化算法提供可解释和可靠的解决方案。不幸的是，这些模型没有利用数据，因此达不到最先进的深度学习模型。这项研究将解决增强深度学习方法的可解释性和可靠性的问题，并在应用此类学习方法时改善公共安全。此外，该项目将为参与的学生提供宝贵的教育机会。参与者将获得逆问题，优化和机器学习方面的知识，这些知识是适用于学术界，政府和工业界的可转移技能。该项目旨在开发一个框架，将优化算法的可解释性和可靠性与DNN的设计和训练相结合。主要关注的是隐式网络，这是一种DNN，它通过固定点或最优性条件隐式地确定其输出，而不是像传统DNN那样具有固定层数的固定数量的计算。将优化算法集成到隐式网络中称为隐式学习优化（L2O）网络。隐式L2O网络有可能克服传统DNN的局限性，包括缺乏可靠性和可解释性。然而，训练和设计隐式L2O模型带来了额外的挑战，阻碍了它们的广泛采用。为了应对这些挑战，该研究旨在开发一个通用的隐含L2O框架。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。