CAREER: New Frontiers in Bayesian Deep Learning

该奖项全部或部分由《2021年美国救援计划法案》（公法117-2）资助。计算机在具有许多移动部件和不断变化的环境的安全关键环境中提供帮助——在新的交通环境中驾驶车辆，对不同分辨率获得的医学图像进行分类，在崎岖的地形上操纵机器人系统，以及设计救生药物。这样的任务需要仔细地表示不确定性，这样我们就可以保护自己免受罕见但代价高昂的错误，并检测我们是否在标准参数之外操作。如果我们能将先验知识融入到我们对不确定性的描述中，我们就能进一步提高表现。例如，我们可能认为，如果图像被旋转或翻译，图像的标签不应该改变。结合这些知识可以从少量的信息中有效地学习，以及更高的准确性和可靠性，通常可以区分哪些问题可以解决，哪些问题不能解决。这项研究使得在从数据中学习的复杂模型中表示不确定性成为可能，同时指定详细的先验知识，提供对不断变化的环境的特殊弹性。它还显著减少了鲁棒不确定性表示所需的计算量，从而使计算机模型能够以更低的成本更快、更可靠地量化不确定性。我们的系统可以对不确定性进行编码，并有效地将知识转移到新的环境中，从而促进准确的医疗诊断、可靠的基础设施和挽救生命的科学发现。我们通过这项工作发展的科学创新将被纳入流行的教育教科书，并作为推广活动项目的基础。这项研究的目标是让贝叶斯深度学习指导深度学习社区的研究轨迹，用原则性的方法来解决基础问题，比如如何管理归纳偏差和灵活性之间的权衡。这个目标对任何预测任务都有影响，提高准确性和健壮性，同时为决策提供可靠的不确定性估计。本研究有三个关键部分，它们形成了天然的协同作用。第一个推力追求新的先验，将灵活性与有用的归纳偏差结合起来，并提供良好的分布外泛化。这些先验提供了一种机制，可以将高级概念编码为先验分布，例如局部性、独立性和对称性，而不限制模型的灵活性。为了实现这些先验的好处，下一个重点是精确的近似推理程序，受到哈密顿蒙特卡罗和深度集成的启发。最后一个重点是在医学成像、自动驾驶和蛋白质设计方面的应用。该奖项旨在表彰对社会有广泛兴趣的项目，包括科学和公共政策应用，以及包括学校教学、开放教育资源、主要开源图书馆、教科书编写、本科生实习、暑期学校、专题讨论会和与当地教育机构合作在内的外展项目。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Computers assist in safety-critical settings, with many moving parts and changing environments — driving vehicles in new traffic environments, classifying medical images acquired with different resolutions, manipulating robotic systems over rugged terrain, and designing life-saving pharmaceuticals. Such tasks require a careful representation of uncertainty so that we can protect ourselves against rare but costly mistakes and detect if we are operating outside of standard parameters. We can further improve performance if we can incorporate prior knowledge into how we represent uncertainty. For example, we may believe that the label of an image should not change if the image is rotated or translated. Incorporating this knowledge enables efficient learning from a small amount of information, as well as greater accuracy and reliability, often making the difference between which problems can be solved and which cannot. This research makes it possible to represent uncertainty in sophisticated models that learn from data, while specifying detailed prior knowledge, providing particular resilience to changing environments. It also significantly reduces the computations needed for a robust uncertainty representation, leading to computer models that can quantify uncertainty more quickly and reliably, at lower cost. With systems that can encode uncertainty and efficiently transfer knowledge to new environments, we facilitate accurate medical diagnoses, reliable infrastructure, and life-saving scientific discoveries. The scientific innovations we develop through this work will be included in popular educational textbooks, and used as the basis for projects in outreach initiatives.The goal of this research is to have Bayesian deep learning guide the research trajectory of the deep learning community at large, with principled approaches to foundational questions, such as how to manage the trade-off between inductive biases and flexibility. This goal has implications for essentially any predictive task, improving accuracy and robustness, while providing reliable uncertainty estimates for decision making. There are three key parts to this research, which form a natural synergy. The first thrust pursues new priors that combine flexibility with useful inductive biases, and provide good out-of-distribution generalization. These priors provide a mechanism for encoding high level concepts into prior distributions, such as locality, independencies, and symmetries, without constraining model flexibility. To realize the benefits of these priors, the next thrust focuses on accurate approximate inference procedures, inspired by Hamiltonian Monte Carlo and deep ensembling. The last thrust pursues applications in medical imaging, autonomous driving, and protein design. This award pursues initiatives that are of broad interest to society, including scientific and public policy applications, and outreach initiatives that include school teaching, open educational resources, major open source libraries, textbook writing, undergraduate internships, summer schools, symposia, and collaborations with local educational institutions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.