CAREER: Score-Based Diffusion Models for Probabilistic Forecasting of Weather and Climate

该项目将开发用于估计恶劣天气和气候事件风险的机器学习方法。估计高维数据中罕见事件的概率是数据科学中的一个基本问题。生成式人工智能的进步将用于开发新的数据驱动的风险建模计算方法，并将这些方法应用于天气应用。特别是，这些模式将用于预报太阳辐照度和降水，这两项应用对夏威夷等热带岛屿特别重要。估计太阳能发电的快速变化（斜坡事件）的风险对于管理能源网络是必要的，因为可变可再生能源的快速增长，洪水每年仅在美国就造成数百人死亡和数十亿美元的财产损失。这项研究将与一项教育推广计划相结合，该计划包括为高中生开设的夏季数据科学课程，以及与当地K-12教师分享数据科学教材的研讨会。生成式人工智能方法在文本到图像模型、图像超分辨率和视频预测方面取得了快速进展。关键的发展是用于学习高维数据（如图像和视频）上的联合概率分布的神经网络模型。其中包括基于分数的扩散模型，它解决了以前生成模型的许多局限性，包括其他基于流的模型、自回归模型、变分自编码器和生成对抗网络。该项目将研究和改进基于分数的扩散模型的能力，以有效地从有限的训练数据中学习罕见事件的概率。应用于太阳辐照度和降水预报将作为激励案例研究，因为这些问题需要对时空天气数据的联合分布进行建模。实验将利用来自大气变量数值模拟的现有数据以及来自卫星和地面传感器网络的观测数据。该项目开发的机器学习方法将补充现有的基于物理的数值天气预报模型，提供具有更高计算速度、时空分辨率和概率预测精度的特定地点预报。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project will develop machine learning methods for estimating the risk of adverse weather and climate events. Estimating the probability of rare events in high-dimensional data is a fundamental problem in data science. Advances in generative artificial intelligence will be used to develop new data-driven computational methods for modeling risk and apply these methods to weather applications. In particular, these models will be applied to forecasting solar irradiance and precipitation, two applications that are particularly important for tropical islands such as Hawaii. Estimating the risk of rapid changes in solar power generation (ramp events) is necessary for managing energy grids that are seeing a rapid increase in variable renewable sources, and floods claim hundreds of lives and billions in property damage each year in the United States alone. This research will be paired with an educational outreach program that includes a summer data science course for high school students and a workshop to share data science teaching materials with local K-12 teachers.Generative artificial intelligence methods have led to rapid progress in text-to-image models, image super-resolution, and video prediction. The key development is in the neural network models used to learn joint probability distributions over high-dimensional data such as images and video. These include score-based diffusion models, which solve many of the limitations of previous generative models including other flow-based models, autoregressive models, variational autoencoders, and generative adversarial networks. This project will investigate and improve the ability of score-based diffusion models to efficiently learn the probability of rare events from finite training data. Applications to solar irradiance and precipitation forecasting will serve as motivating case-studies, as these problems require modeling the joint distributions over spatiotemporal weather data. Experiments will leverage existing data from numerical simulations of atmospheric variables and observations from satellites and ground-based sensor networks. The machine learning methods developed by this project will complement existing physics-based numerical weather prediction models by providing location-specific forecasts with increased computational speed, spatiotemporal resolution, and probabilistic prediction accuracy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.