Method for improving forecast statistics in large-scale variational data assimilation problems

在过去二十年中，数据同化方法已成为分析和预测大气和海洋中复杂现象的重要工具。这些方法最为人所知的应用，大概是中短期天气预报，它已成为指导我们日常生活的日常服务。从更广泛的角度来看，这种预测具有巨大的经济和安全价值。最新的例子可能包括火山灰云在大气中的扩散和海洋中的石油泄漏跟踪。精确监测和预测这种不幸事件的重要性几乎不需要任何进一步的宣传。数据同化方法基本上是一种数学方法，可以将大气和海洋流动的数学模型、观测数据（由气象站或卫星测量）以及反映我们在几十年的观测和分析过程中积累的对这些现象的经验的一些额外信息结合起来。气象学和海洋学中数据同化问题的一个显着特征是它们的大尺度，这意味着流动模型包含大量变量，比如$\sim 10^9$或更多。这就是为什么，尽管计算能力有了非常显著的提高，但研究人员和实践者在解决这些问题时，对合适的数学和算法工具的选择仍然非常有限。在几种可行的方法来解决这些问题的变分数据同化方法称为“4D-Var”是在一些主要的业务中心，如英国气象局，ECMWF，气象法国等，而这种方法产生的预测通常是合理的好，目前仍然没有解决的问题，评估预测协方差矩阵的4D-Var框架。这个协方差矩阵需要评估预测的好坏和可用性;没有这样的评估，预测本身可能没有什么价值。当观测到的现象是高度非线性的（在大多数情况下都是这种情况）时，目前采用的技术可能无法提供合理质量的预测协方差矩阵。让我们再次强调，在数学和统计学中有一些方法能够在原则上解决这个问题;正是问题的维度使得它们中的大多数无法使用。因此，本研究的主要目标是开发一种计算上可行的技术，用于评估4D-Var框架中预测协方差矩阵的准确近似。实现这一目标的建议方法将基于申请人最近在高影响力计算期刊上发表的最新结果。它代表了用于解决数据同化问题的确定性和统计（贝叶斯）方法之间的良好平衡，并有可能在实际应用中最终投入使用。这一潜力已得到已发表论文的评审者和该领域一些主要专家的私人通信的证实。然而，在目前的状态下，该方法需要理论和算法升级，更重要的是，需要用现实的大规模模型进行更实质性的验证，例如NEMO海洋模型，该模型正在迅速流行。所有这些步骤都将在拟议的研究中进行。

Over the past two decades, methods of data assimilation have become vital tools for analysis and prediction of complex phenomena in the atmosphere and the ocean. Probably, the most well known application of these methods is the short- and medium-range weather prediction which has become a routine service guiding our everyday life. A wider outlook reveals the great economic and security value of such forecasts. The latest examples may include the volcano ash cloud dispersion in the atmosphere and oil spill tracking in the ocean. The importance of precise monitoring and prediction of such unfortunate events hardly requires any further promotion. The data assimilation methods are, essentially, mathematical methods which allow the combination of mathematical models of the atmospheric and oceanic flows, observation data (measured by weather stations or by satellites) and some additional information which reflects our experience with these phenomena, accumulated during decades of observations and analysis. A distinguishing feature of the data assimilation problems in meteorology and oceanography is their large-scale, which means that flow models contain a huge number of variables, say $\sim 10^9$ and more. That is why, despite a very significant increase in computational power, the researchers and practitioners still have a very limited choice of suitable mathematical and, subsequently, algorithmic tools for solving these problems. Among few methods feasible for solving these problems the variational data assimilation method called '4D-Var' is the preferred method implemented at some major operational centers, such as the UK Met Office, ECMWF, Meteo France, etc. While the forecasts produced by this method are usually reasonably good, the problem of evaluating the forecast covariance matrix in the 4D-Var framework currently remains unsolved. This covariance matrix is needed to assess how good and usable the forecast is; without such assessment the forecast itself may be of littlevalue. The technique currently adopted may not provide the forecast covariance matrix of a reasonable quality when the observed phenomena are highly nonlinear (which is the case in most circumstances). Let us stress again that there are methods in mathematics and statistics capable of solving this problem in principle; it is the problem's dimensionality that makes most of them unusable. Therefore, the main objective of this research is to develop a computationally feasible technique for evaluating an accurate approximation of the forecast covariance matrix in the 4D-Var framework. The suggested methodology to achieve this objective will be based on the latest results recently published by the applicant in high impact computational journals. It represents a fine balance between deterministic and statistical (Bayesian) methods used for solving data assimilation problems and has a potential for the eventual operational use in real-life applications. This potential has been confirmed both by the reviewers of the published papers and in private communications involving some leading experts in the field. However, in its current state the method needs a theoretical and algorithmic upgrade and, importantly, more substantial verification with a realistic large-scale models, such as the NEMO ocean model, which is rapidly becoming popular. All these steps will be conducted in the proposed research.