Generalization properties of feed-forward neural networks trained on Lorenz systems
Generalization properties of feed-forward neural networks trained on Lorenz systems
复制标题
在洛伦兹系统上训练的前馈神经网络的泛化特性
DOI：
10.5194/npg-26-381-2019
复制
发表时间：
2019
影响因子：
2.2
通讯作者：
G. Messori
中科院分区：
文献类型：
--
作者：
S. Scher;G. Messori
摘要。当神经网络被提供涵盖系统相空间所有相关区域的训练数据时，它能够近似混沌动力系统。然而，许多实际应用偏离了这种理想化的情形。在此，我们研究前馈神经网络以下能力：（1）从不完全的训练数据中学习动力系统的行为；（2）学习外部强迫对动力学的影响。气候科学是一个现实世界的例子，这些问题可能与之相关：它涉及一个受外部强迫的非平稳混沌系统，其行为仅通过相对较短的数据序列为人所知。我们的分析是针对洛伦兹63和洛伦兹95模型进行的。我们表明，对于洛伦兹63系统，在仅覆盖系统相空间一部分的数据上训练的神经网络难以对训练未涵盖的区域做出准确的短期预测。此外，当进行长时间连续预测时，这些网络难以重现探索训练数据中未出现区域的轨迹，除非在训练期间只有小部分区域被遗漏。我们发现这是由于神经网络为训练数据中的相空间每个区域学习了局部映射，而非全局映射。这表现为网络的某些部分仅学习相空间的特定部分。相比之下，对于洛伦兹95系统，网络能够成功地推广到训练数据中未出现的相空间新区域。我们还发现，网络能够学习外部强迫的影响，但仅当在训练中给定相对较大范围的强迫时才行。这些结果指出了前馈神经网络在给定有限初始信息的情况下推广系统行为的潜在局限性。因此，在为实际应用设计合适的训练 - 测试划分时必须给予高度关注。
Abstract. Neural networks are able to approximate chaotic dynamical systems when provided with training data that cover all relevant regions of the system's phase space. However, many practical applications diverge from this idealized scenario. Here, we investigate the ability of feed-forward neural networks to (1) learn
the behavior of dynamical systems from incomplete training data
and (2) learn the influence of an external forcing on the dynamics. Climate science is a real-world example where these questions may be relevant: it is concerned with a non-stationary chaotic system subject to external forcing and whose behavior is known only through comparatively short data series. Our analysis is performed on the Lorenz63 and Lorenz95 models. We show that for the Lorenz63 system, neural networks trained on data covering only part of the system's phase space struggle to make skillful short-term forecasts in the regions excluded from the training. Additionally, when making long series of consecutive forecasts, the networks struggle to reproduce trajectories exploring regions beyond those seen in the training data, except for cases where only small parts are left out during training. We find this is due to the neural network learning a localized mapping for each region of phase space in the training data rather than a global mapping. This manifests itself in that parts of the networks learn only particular parts of the phase space. In contrast, for the Lorenz95 system the networks succeed in generalizing to new parts of the phase space not seen in the training data. We also find that the networks are able to learn the influence of an external forcing, but only when given relatively large ranges of the forcing in the training. These results point to potential limitations of feed-forward neural networks in generalizing a system's behavior given limited initial information. Much attention must therefore be given to designing appropriate train-test splits for real-world applications.
DOI：
10.1073/pnas.1810286115
发表时间：
2018-09-25
影响因子：
11.1
作者：
Rasp S;Pritchard MS;Gentine P
通讯作者：
Gentine P