Multi-Resolution Active Learning of Fourier Neural Operators
Multi-Resolution Active Learning of Fourier Neural Operators
复制标题
DOI：
10.48550/arxiv.2309.16971
复制
发表时间：
2023-09
期刊：
影响因子：
--
通讯作者：
Shibo Li;Xin Yu;Wei W. Xing;Mike Kirby;Akil Narayan;Shandian Zhe
中科院分区：
文献类型：
--
作者：
Shibo Li;Xin Yu;Wei W. Xing;Mike Kirby;Akil Narayan;Shandian Zhe
傅立叶神经算子（FNO）是一种流行的算子学习框架。它不仅在许多任务中实现了最先进的性能，而且在训练和预测方面也非常高效。然而，收集 FNO 训练数据在实践中可能是一个成本高昂的瓶颈，因为它通常需要昂贵的物理模拟。为了克服这个问题，我们提出了FNO的多分辨率主动学习（MRA-FNO），它可以动态选择输入函数和分辨率，在优化学习效率的同时尽可能降低数据成本。具体来说，我们提出了一种概率多分辨率 FNO，并使用集成蒙特卡罗开发了一种有效的后验推理算法。为了进行主动学习，我们最大化效用成本比作为获取函数，以在每一步获取新的示例和解决方案。我们使用矩匹配和矩阵行列式引理来实现易于处理、高效的效用计算。此外，我们开发了一个成本退火框架，以避免在早期阶段过度惩罚高分辨率查询。当分辨率之间的成本差异显着时，过度惩罚会很严重，这使得主动学习经常陷入低分辨率查询和较差的性能。我们的方法克服了这个问题，并适用于一般的多保真主动学习和优化问题。我们已经在几个基准算子学习任务中展示了我们的方法的优势。该代码可在 https://github.com/shib0li/MRA-FNO 获取。
Fourier Neural Operator (FNO) is a popular operator learning framework. It not only achieves the state-of-the-art performance in many tasks, but also is efficient in training and prediction. However, collecting training data for the FNO can be a costly bottleneck in practice, because it often demands expensive physical simulations. To overcome this problem, we propose Multi-Resolution Active learning of FNO (MRA-FNO), which can dynamically select the input functions and resolutions to lower the data cost as much as possible while optimizing the learning efficiency. Specifically, we propose a probabilistic multi-resolution FNO and use ensemble Monte-Carlo to develop an effective posterior inference algorithm. To conduct active learning, we maximize a utility-cost ratio as the acquisition function to acquire new examples and resolutions at each step. We use moment matching and the matrix determinant lemma to enable tractable, efficient utility computation. Furthermore, we develop a cost annealing framework to avoid over-penalizing high-resolution queries at the early stage. The over-penalization is severe when the cost difference is significant between the resolutions, which renders active learning often stuck at low-resolution queries and inferior performance. Our method overcomes this problem and applies to general multi-fidelity active learning and optimization problems. We have shown the advantage of our method in several benchmark operator learning tasks. The code is available at https://github.com/shib0li/MRA-FNO.