A Fortran-Keras Deep Learning Bridge for Scientific Computing
A Fortran-Keras Deep Learning Bridge for Scientific Computing
复制标题
DOI：
10.1155/2020/8888811
复制
发表时间：
2020-08-28
影响因子：
--
通讯作者：
Baldi, Pierre
中科院分区：
文献类型：
--
作者：
Ott, Jordan;Pritchard, Mike;Baldi, Pierre
实现人工神经网络通常通过高级编程语言（如Python）和易于使用的深度学习库（如Keras）实现。这些软件库预装了各种网络架构，提供自动微分，并支持GPU进行快速高效的计算。因此，深度学习从业者将倾向于在Python中训练神经网络模型，因为这些工具很容易获得。然而，许多大型科学计算项目都是用Fortran编写的，这使得它很难与现代深度学习方法集成。为了缓解这个问题，我们引入了一个软件库，Fortran-Keras Bridge（FKB）。这种双向桥梁将深度学习资源丰富的环境与资源稀缺的环境连接起来。本文介绍了FKB提供的几个独特功能，如可定制的层，损失函数和网络集成。本文最后以一个案例研究为例，应用FKB来解决有关全球气候模拟实验方法的鲁棒性的公开问题，其中亚网格物理学被外包给深度神经网络仿真器。在这种情况下，FKB可以对100多个亚网格云和辐射物理学候选模型进行超参数搜索，这些模型最初在Keras中实现，然后在Fortran中传输和使用。这样的过程允许模型的涌现行为被评估，即，当拟合缺陷与明确的行星尺度流体动力学耦合时。结果揭示了离线验证错误和在线性能之间的一种以前未被认识到的强关系，其中优化器的选择被证明是意想不到的关键。这反过来又揭示了许多新的神经网络架构，这些架构在气候模型稳定性方面产生了相当大的改进，包括一些减少了误差的架构，对于一个特别具有挑战性的训练数据集。
Implementing artificial neural networks is commonly achieved via high-level programming languages such as Python and easy-to-use deep learning libraries such as Keras. These software libraries come preloaded with a variety of network architectures, provide autodifferentiation, and support GPUs for fast and efficient computation. As a result, a deep learning practitioner will favor training a neural network model in Python, where these tools are readily available. However, many large-scale scientific computation projects are written in Fortran, making it difficult to integrate with modern deep learning methods. To alleviate this problem, we introduce a software library, the Fortran-Keras Bridge (FKB). This two-way bridge connects environments where deep learning resources are plentiful with those where they are scarce. The paper describes several unique features offered by FKB, such as customizable layers, loss functions, and network ensembles. The paper concludes with a case study that applies FKB to address open questions about the robustness of an experimental approach to global climate simulation, in which subgrid physics are outsourced to deep neural network emulators. In this context, FKB enables a hyperparameter search of one hundred plus candidate models of subgrid cloud and radiation physics, initially implemented in Keras, to be transferred and used in Fortran. Such a process allows the model's emergent behavior to be assessed, i.e., when fit imperfections are coupled to explicit planetary-scale fluid dynamics. The results reveal a previously unrecognized strong relationship between offline validation error and online performance, in which the choice of the optimizer proves unexpectedly critical. This in turn reveals many new neural network architectures that produce considerable improvements in climate model stability including some with reduced error, for an especially challenging training dataset.