EAGER: Search-Accelerated Markov Chain Monte Carlo Algorithms for Bayesian Neural Networks and Trillion-Dimensional Problems

EAGER：贝叶斯神经网络和万亿维问题的搜索加速马尔可夫链蒙特卡罗算法

• 批准号: 2404989
• 负责人: Peter Behroozi
• 金额: $ 16.33万
• 依托单位: University of Arizona
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-04-01 至 2025-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2404989&HistoricalAwards=false
• 关键词: EAGER; Search; Accelerated; Markov; Chain

Machine learning with deep neural networks occurs in every scientific field. In astronomy, machine learning (ML) applications are wide-ranging. Advanced ML methods involve large numbers of tunable variables, in the range of millions to trillions of parameters. Current methods for tuning these parameters can find a single best-fitting model but are unable to produce the range of ML models that match observations. Knowing the range of uncertainties for a given method is of fundamental importance in science. More broadly, there are many areas beyond science where knowing the range of uncertainties is important (e.g., driverless cars, medical, and military applications). This proposal introduces a new algorithm that will explore uncertainties for trillion-dimensional models and beyond (encompassing the dimensionality of current large neural networks). For such trillion-dimensional parameter spaces, the method would be five hundred times faster than the best previous approaches. This method could revolutionize machine learning across scientific and commercial fields. The ability to show that all reasonable neural networks give similar results would directly address present problems of robustness and reproducibility, as well as rigorously quantify model uncertainties. This proposal also involves direct broader impact work with veterans transitioning from the military to college. This proposal would advance the frontiers of sampling algorithm performance, with typical O(log D) or better scaling with dimensionality, as well as provide new knowledge about the geometry of high-dimensional posterior distributions, including those for deep neural networks. Advancing sampling algorithm performance would allow a broad range of problems to be addressed in astronomy that would otherwise be computationally intractable, especially reconstruction problems (e.g., field-level inference for cosmology or blended source reconstruction) and high-dimensional modeling (e.g., modeling the joint distribution of physical properties of multiple galaxies simultaneously, given their luminosity, spatial, and redshift distributions). Advancing understanding of the geometry of high-dimensional posterior distributions would enable the development of more optimized algorithms for exploration and sampling, particularly for advanced neural networks, further reducing the barriers to robust posterior distributions. The results of this proposal will be released as an open-source implementation of the algorithm as well as an open-access accompanying paper.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度神经网络的机器学习在每个科学领域都有应用。在天文学中，机器学习（ML）的应用非常广泛。高级机器学习方法涉及大量可调变量，范围从数百万到数万亿个参数。目前用于调整这些参数的方法可以找到单个最佳拟合模型，但无法产生与观测值匹配的ML模型范围。了解给定方法的不确定度范围在科学中是至关重要的。更广泛地说，在科学以外的许多领域，了解不确定性的范围很重要（例如，无人驾驶汽车、医疗和军事应用）。该提案引入了一种新的算法，该算法将探索万亿维模型及以上（包括当前大型神经网络的维度）的不确定性。对于这样的万亿维参数空间，该方法将比以前最好的方法快500倍。这种方法可以在科学和商业领域彻底改变机器学习。证明所有合理的神经网络给出相似结果的能力将直接解决当前的稳健性和可重复性问题，以及严格量化模型不确定性的能力。这项提案还涉及到对退伍军人从军队过渡到大学的直接影响工作。该建议将推进采样算法性能的前沿，具有典型的O（log D）或更好的维数缩放，并提供有关高维后验分布几何的新知识，包括深度神经网络的后验分布。先进的采样算法性能将允许解决天文学中广泛的问题，否则这些问题在计算上是难以解决的，特别是重建问题（例如，宇宙学的场级推断或混合源重建）和高维建模（例如，同时建模多个星系的物理特性的联合分布，给定它们的光度，空间和红移分布）。加深对高维后验分布几何的理解，将有助于开发更优化的勘探和抽样算法，特别是对于高级神经网络，进一步减少鲁棒后验分布的障碍。该提案的结果将作为该算法的开源实现以及开放获取的随附论文发布。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。