CAREER: New Frontiers in Bayesian Deep Learning

职业：贝叶斯深度学习的新领域

• 批准号: 2145492
• 负责人: Andrew Wilson
• 金额: $ 48.52万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-05-01 至 2027-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2145492&HistoricalAwards=false
• 关键词: CAREER; New; Frontiers; Bayesian; Deep

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Computers assist in safety-critical settings, with many moving parts and changing environments — driving vehicles in new traffic environments, classifying medical images acquired with different resolutions, manipulating robotic systems over rugged terrain, and designing life-saving pharmaceuticals. Such tasks require a careful representation of uncertainty so that we can protect ourselves against rare but costly mistakes and detect if we are operating outside of standard parameters. We can further improve performance if we can incorporate prior knowledge into how we represent uncertainty. For example, we may believe that the label of an image should not change if the image is rotated or translated. Incorporating this knowledge enables efficient learning from a small amount of information, as well as greater accuracy and reliability, often making the difference between which problems can be solved and which cannot. This research makes it possible to represent uncertainty in sophisticated models that learn from data, while specifying detailed prior knowledge, providing particular resilience to changing environments. It also significantly reduces the computations needed for a robust uncertainty representation, leading to computer models that can quantify uncertainty more quickly and reliably, at lower cost. With systems that can encode uncertainty and efficiently transfer knowledge to new environments, we facilitate accurate medical diagnoses, reliable infrastructure, and life-saving scientific discoveries. The scientific innovations we develop through this work will be included in popular educational textbooks, and used as the basis for projects in outreach initiatives.The goal of this research is to have Bayesian deep learning guide the research trajectory of the deep learning community at large, with principled approaches to foundational questions, such as how to manage the trade-off between inductive biases and flexibility. This goal has implications for essentially any predictive task, improving accuracy and robustness, while providing reliable uncertainty estimates for decision making. There are three key parts to this research, which form a natural synergy. The first thrust pursues new priors that combine flexibility with useful inductive biases, and provide good out-of-distribution generalization. These priors provide a mechanism for encoding high level concepts into prior distributions, such as locality, independencies, and symmetries, without constraining model flexibility. To realize the benefits of these priors, the next thrust focuses on accurate approximate inference procedures, inspired by Hamiltonian Monte Carlo and deep ensembling. The last thrust pursues applications in medical imaging, autonomous driving, and protein design. This award pursues initiatives that are of broad interest to society, including scientific and public policy applications, and outreach initiatives that include school teaching, open educational resources, major open source libraries, textbook writing, undergraduate internships, summer schools, symposia, and collaborations with local educational institutions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项全部或部分由2021年美国救援计划法案（公法117-2）资助。计算机在安全关键环境中提供帮助，其中有许多移动部件和不断变化的环境-在新的交通环境中驾驶车辆，对不同分辨率的医学图像进行分类，在崎岖的地形上操纵机器人系统，以及设计救生药物。这样的任务需要对不确定性进行仔细的描述，以便我们能够保护自己免受罕见但代价高昂的错误的影响，并检测我们是否在标准参数之外操作。如果我们能够将先验知识融入到我们如何表示不确定性中，我们就可以进一步提高性能。例如，我们可能认为，如果图像被旋转或平移，则图像的标签不应该改变。掌握这些知识可以从少量信息中进行有效学习，以及更高的准确性和可靠性，通常会在哪些问题可以解决和哪些问题不能解决之间产生差异。这项研究使得在从数据中学习的复杂模型中表示不确定性成为可能，同时指定详细的先验知识，为不断变化的环境提供特别的弹性。它还大大减少了稳健的不确定性表示所需的计算，从而使计算机模型能够以更低的成本更快速可靠地量化不确定性。通过能够对不确定性进行编码并将知识有效地转移到新环境中的系统，我们可以促进准确的医疗诊断，可靠的基础设施和挽救生命的科学发现。我们通过这项工作开发的科学创新将包含在流行的教育教科书中，并作为外展计划项目的基础。这项研究的目标是让贝叶斯深度学习指导整个深度学习社区的研究轨迹，用原则性的方法来解决基础问题，例如如何管理归纳偏差和灵活性之间的权衡。这一目标对基本上任何预测任务都有影响，提高了准确性和鲁棒性，同时为决策提供了可靠的不确定性估计。这项研究有三个关键部分，它们形成了自然的协同作用。第一个推力追求新的先验知识，将联合收割机灵活性与有用的归纳偏差相结合，并提供良好的分布外推广。 这些先验提供了一种机制，用于将高级概念编码到先验分布中，例如局部性、独立性和对称性，而不限制模型的灵活性。为了实现这些先验的好处，下一个重点是精确的近似推理程序，灵感来自汉密尔顿蒙特卡罗和深度集成。最后一个目标是在医学成像、自动驾驶和蛋白质设计方面的应用。该奖项追求的是对社会有广泛兴趣的倡议，包括科学和公共政策应用，以及包括学校教学，开放教育资源，主要开源图书馆，教科书编写，本科生实习，暑期学校，研讨会，该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的学术价值和更广泛的影响审查标准。

项目成果

On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification

• DOI: 10.48550/arxiv.2203.16481
• 发表时间: 2022-03
• 期刊: ArXiv
• 作者: Sanyam Kapoor;Wesley J. Maddox;Pavel Izmailov;A. Wilson

Pre-Train Your Loss: Easy Bayesian Transfer Learning with Informative Priors

• DOI: 10.48550/arxiv.2205.10279
• 发表时间: 2022-05
• 期刊: ArXiv
• 作者: Ravid Shwartz-Ziv;Micah Goldblum;Hossein Souri;Sanyam Kapoor;Chen Zhu;Yann LeCun;A. Wilson

Bayesian Optimization with Conformal Prediction Sets

• 发表时间: 2022-10
• 作者: S. Stanton;Wesley J. Maddox;A. Wilson