Combining Machine Learning and Data Assimilation to infer model errors

数据同化(DA)是一门将观测(或数据)中包含的信息与手头系统的先验知识相结合的科学，通常以一组耦合的偏微分方程组的形式存在。这是一种应用于地球科学，特别是气象学和气候科学的贝叶斯推理，在这些科学中，流体动力学定律是已知的，并且必然会被使用。然后，这些知识被转化为一个随时间演变的数值模型，该模型的大小远远大于可用数据量。由于这种模型与观察维度的不匹配，模型对DA的结果起着关键作用，因此可以将其视为模型驱动的过程。DA中使用的地质流体数值模型只是真实大气、海洋或整个气候系统的近似表示。必须考虑由此产生的模型误差，并且已经投入了大量的工作来使DA方法能够以统计方式适应模型误差。真实的模型误差是未知的，并且产生于许多不同的来源，如数值离散化、参数误差和未分辨尺度的存在。尤其是子网格过程对模型的技巧非常关键，并在适当的子网格参数化方案中进行了描述。估计这些方案的形式和它们的参数值对于预报和成功的数据同化都是至关重要的。现有的估计技术在很大程度上依赖于物理直觉和对可用观测的临时使用。然而，从模型观测失配数据中估计模型误差的系统和稳健的方法还不存在。另一方面，近年来可用观测值的不断增加，伴随着同样惊人的计算能力的增长，使得完全由数据驱动的方法成为可能。这场数据驱动的革命主要是由机器学习(ML)技术(如深度神经网络等)的蓬勃发展推动的，这些技术越来越成功地被证明能够从多变量数据集中提取潜在的动力学规律，具有令人印象深刻的预测技能和对复杂行为进行分类的能力。目前，ML和DA算法非常相似：两种方法都在给定一组目标(即观察)的情况下优化参数。优化，或ML术语的训练，需要计算DA中的梯度和伴随项，在ML中称为反向传播。主要不同之处在于，在DA中，模型是作为一组物理c约束明确列出的，而在ML中，只是最近才成熟起来，可以结合我们的物理知识。此外，与发展议程相反，ML没有使用原则性不确定性量化方法。ML和DA的互补性、发展议程在地学领域的成功以及ML在同一领域的光明前景，促使人们寻找它们的适当组合，充分利用它们各自的长处，减轻各自的弱点。建议的PHD研究计划将在这一边界上工作。我们建议使用机器学习来学习在核心动力学模型中没有明确描述的子网格过程的参数化，基于来自DA实验的模型-观测失配数据。然后，这个新的参数化将在模型中实现，并用于执行DA。由于DA的性能与所使用的模型误差是非线性相关的，因此ML可以再次使用所产生的模型-观测失配数据来改进其参数化描述，然后将其用于DA。这一迭代过程，如果定义良好，将会收敛，通过模型改进和模型的高级DA/ML初始化，在环境预测方面带来潜在的突破。

Data assimilation (DA) is the science of combining information contained in observations (or 'data') with prior knowledge of the system at hand, typically in the form of a coupled set of partial differential equations. It is Bayesian Inference applied to the geosciences, especially meteorology and climate science, where the laws of fluid-dynamics are known and bound to be used. This knowledge is then translated into a time-evolving numerical model and the size of the model is vastly larger than the amount of available data. As a consequence of this model-to-observation dimensional mismatch, the models play a critical role on the outcome of DA, that can thus be seen as a model-driven procedure. Numerical models of geo-fluids used in DA are only an approximate representation of the real atmosphere, ocean or the whole climate system. The resulting model error has to be taken into account and a substantial amount of work has been devoted to make DA methods able to accommodate model error in a statistical way. The real model error is unknown, and arises from many different sources, such as numerical discretization, parametric error, and the presence of unresolved scales. Sub-grid processes in particular are very critical to the skill of the models and are described in appropriate sub-grid parametrization schemes. Estimating the form of these schemes and the values of their parameters is of crucial importance, both for prediction and for successful data assimilation. Existing estimation techniques rely heavily on physical intuition and ad-hoc use of available observations. However, systematic and robust methods to estimate model errors from model-observation mismatch data do not exist.On the other hand, in recent times the constant increase of available observations, accompanied by a similarly spectacular growth of the computing power, have made fully data-driven approaches possible. This data-driven revolution has been mostly pushed by the flourishing of machine-learning (ML) techniques (e.g. deep neural networks, among others) that with increasing success have shown to be able to extract the underlying dynamical laws from a multivariate dataset, with impressive predictive skill and capabilities to classify complex behaviors.At present, ML and DA algorithms are quite similar: both approaches optimize parameters given a set of targets (i.e., the observations). The optimization, or the training in ML jargon, requires computing gradients and adjoints in DA, referred to as backpropagation in ML. The major difference is that while in the DA the model is explicitly set out as a set of physical c constraints, in ML is only recently maturing to incorporate our physical knowledge. Furthermore, as opposed to DA, no principled uncertainty quantification is used in ML.The complementarities of ML and DA, the success of DA in the geoscience, and the promising future of ML in the same area, motivates the search for suitable combinations of them that adequately exploit each of their strength and mitigate each of their weaknesses. The proposed PhD research program is will work at this boundary.We propose to use machine learning to "learn" the parametrization of sub-grid processes that are not explicitly described in the core dynamical model, based on model-observation mismatch data from DA experiments. This new parametrization will then be implemented in the model and used to perform DA. Since the DA performance is nonlinearly related to the model error used, the resulting model-observation mismatch data can again we used by ML to improve its parameterization description, which can then be used in DA. This iterative process, if well defined, will converge, leading to a potential breakthrough in environmental prediction via both model improvement and superior DA/ML initialization of the models.