CAREER: New Frontiers in Bayesian Deep Learning

该奖项的全部或部分资金来自《2021年美国救援计划法案》(公法117-2)。计算机协助安全关键环境，包括许多移动部件和不断变化的环境--在新的交通环境中驾驶车辆，对不同分辨率获取的医学图像进行分类，在崎岖的地形上操纵机器人系统，以及设计救命药物。这样的任务需要仔细地表示不确定性，以便我们能够保护自己免受罕见但代价高昂的错误的影响，并检测我们是否在标准参数之外操作。如果我们能够将先验知识融入到我们表示不确定性的方式中，我们可以进一步提高性能。例如，我们可能认为，如果图像被旋转或平移，图像的标签不应该改变。融入这些知识可以有效地从少量信息中学习，以及更高的准确性和可靠性，这往往决定了哪些问题可以解决，哪些不能解决。这项研究使得在复杂的模型中表现不确定性成为可能，这些模型从数据中学习，同时指定详细的先验知识，提供对不断变化的环境的特殊弹性。它还显著减少了稳健的不确定性表示所需的计算量，导致了能够以更低的成本更快、更可靠地量化不确定性的计算机模型。通过能够对不确定性进行编码并将知识高效地传输到新环境的系统，我们促进了准确的医疗诊断、可靠的基础设施和拯救生命的科学发现。我们通过这项工作开发的科学创新将被纳入流行的教育教科书，并作为推广倡议项目的基础。本研究的目标是让贝叶斯深度学习指导整个深度学习社区的研究轨迹，并以原则性方法解决基本问题，如如何管理归纳偏见和灵活性之间的权衡。这一目标基本上适用于任何预测性任务，提高了准确性和稳健性，同时为决策提供了可靠的不确定性估计。这项研究有三个关键部分，这三个部分形成了自然的协同效应。第一个推力追求将灵活性与有用的归纳偏差相结合的新先验，并提供良好的分布外泛化。这些先验提供了一种机制，用于将高级概念编码到先前的分布中，例如局部性、独立性和对称性，而不会限制模型的灵活性。为了实现这些先验的好处，下一个重点是受哈密尔顿蒙特卡罗和深度集成的启发，专注于准确的近似推理程序。最后一个重点是在医学成像、自动驾驶和蛋白质设计方面的应用。该奖项追求社会广泛关注的计划，包括科学和公共政策应用，以及推广计划，包括学校教学、开放教育资源、主要开源图书馆、教科书编写、本科生实习、暑期学校、研讨会以及与当地教育机构的合作。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Computers assist in safety-critical settings, with many moving parts and changing environments — driving vehicles in new traffic environments, classifying medical images acquired with different resolutions, manipulating robotic systems over rugged terrain, and designing life-saving pharmaceuticals. Such tasks require a careful representation of uncertainty so that we can protect ourselves against rare but costly mistakes and detect if we are operating outside of standard parameters. We can further improve performance if we can incorporate prior knowledge into how we represent uncertainty. For example, we may believe that the label of an image should not change if the image is rotated or translated. Incorporating this knowledge enables efficient learning from a small amount of information, as well as greater accuracy and reliability, often making the difference between which problems can be solved and which cannot. This research makes it possible to represent uncertainty in sophisticated models that learn from data, while specifying detailed prior knowledge, providing particular resilience to changing environments. It also significantly reduces the computations needed for a robust uncertainty representation, leading to computer models that can quantify uncertainty more quickly and reliably, at lower cost. With systems that can encode uncertainty and efficiently transfer knowledge to new environments, we facilitate accurate medical diagnoses, reliable infrastructure, and life-saving scientific discoveries. The scientific innovations we develop through this work will be included in popular educational textbooks, and used as the basis for projects in outreach initiatives.The goal of this research is to have Bayesian deep learning guide the research trajectory of the deep learning community at large, with principled approaches to foundational questions, such as how to manage the trade-off between inductive biases and flexibility. This goal has implications for essentially any predictive task, improving accuracy and robustness, while providing reliable uncertainty estimates for decision making. There are three key parts to this research, which form a natural synergy. The first thrust pursues new priors that combine flexibility with useful inductive biases, and provide good out-of-distribution generalization. These priors provide a mechanism for encoding high level concepts into prior distributions, such as locality, independencies, and symmetries, without constraining model flexibility. To realize the benefits of these priors, the next thrust focuses on accurate approximate inference procedures, inspired by Hamiltonian Monte Carlo and deep ensembling. The last thrust pursues applications in medical imaging, autonomous driving, and protein design. This award pursues initiatives that are of broad interest to society, including scientific and public policy applications, and outreach initiatives that include school teaching, open educational resources, major open source libraries, textbook writing, undergraduate internships, summer schools, symposia, and collaborations with local educational institutions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.