Variance-based Sensitivity Analysis of the Relationship between Model Parameters and Spatial Features of Forecasts

数值模型需要调整（“模型调整”），以考虑由于对物理过程、算法近似、未解决的过程及其与整体模型的集成/相互作用的不完全了解而产生的不确定性。为此，预测通常被总结为标量（例如，跨预测域的平均值），根据它来检查模型参数的影响。然而，最近空间验证方法的出现要求关注预测的空间特征，以及这些特征如何受到模型参数的影响。本研究旨在开发一种解决这一问题的方法。这样一个框架是必要和及时的，因为它的成分已经出现。虽然空间验证的主题是新的和快速变化的，它的许多元素已经结晶成固定装置。例如，现在很清楚，降水的验证必须考虑到预报域中的“对象”，并且预测误差的评估必须处理这些对象的大小、空间位置和方向上的误差。本研究考察了可以通过pi开发的用于空间验证的统计方法（例如聚类）客观识别的特征/对象。评估模式参数对具体预报特征影响的技术也在迅速发展。本研究的重点是敏感性分析的一类方法，称为基于方差（或全局）。pi已经将这种方法应用于许多例子，包括Lorenz的63模式，以及从一个可操作的中尺度天气预报模式中获得的降水预报的一些非空间和半空间特征。本文将基于方差的敏感性分析方法应用于空间预测；该方法非常适合于多个模型参数和多个预测特征之间存在非线性关系和相互作用的问题。该方法将在海洋/大气耦合中尺度预报系统（COAMPS）和天气研究与预报模式（WRF）上进行试验。知识价值：这项工作的知识价值在于扩展了模型调整的概念，使其与空间验证的基本原则保持一致。该框架将通过允许用户调整模型参数以具有理想的空间特征来改进预测。pi对空间技术的发展以及适用于该问题的灵敏度分析方法做出了贡献。更广泛的影响：研究的更广泛的影响是多方面的。参与该项目的学生将成为数值预测模型中一些最复杂主题的专家：空间验证，灵敏度分析和模型调整。这个高度跨学科的框架本身具有普遍的适用性，因为它可以用于改进任何涉及物理过程不精确参数化或物理常数不精确知识的数值模型的预测，包括天气，气候，海洋等。鉴于空间验证正在成为未来预测的首选验证方法，该方法将得到广泛应用。该方法将以期刊文章的形式传播，计算机代码将包括在国家大气研究中心（NCAR）制作的验证R包中。

Numerical models require adjustments ("model tuning") to allow for uncertainty due to incomplete knowledge of physical processes, algorithmic approximations, unresolved processes and their integration/interaction with the overall model. To that end, forecasts are commonly summarized into scalar quantities (e.g., mean across forecast domain), in terms of which the effect of the model parameters is examined. However, recent advent of spatial verification methods calls for a focus on spatial features of the forecasts, and how these features are affected by model parameters. This research aims to develop a methodology for addressing this problem.Such a framework is necessary and timely because its ingredients have already emerged. Although the topic of spatial verification is new and rapidly changing, many of its elements have crystallized into fixtures. For instance, it is now clear that verification of precipitation must account for "objects" within the forecast field, and that the assessment of forecast errors must address errors in magnitude, spatial placement, and orientation of such objects. This study examines features/objects which can be objectively identified with statistical methods (e.g., clustering) developed by the PIs for spatial verification.Techniques for assessing the effect of model parameters on specific forecast features are also in rapid development. This study focuses on a class of methods for sensitivity analysis, called variance-based (or global). The PIs have already applied this method to a number of examples, including the Lorenz's 63 model, and some non-spatial and semi-spatial features of precipitation forecasts from an operational mesoscale weather forecast model. Here, a variance-based sensitivity analysis method will be adapted to spatial forecasts; the method is ideally suited to problems where there are nonlinear relationships and interactions between multiple model parameters and multiple forecast features. The method will be tested on Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) and Weather Research and Forecasting model (WRF).Intellectual Merit: The intellectual merit of the work is in extending the notion of model tuning to align it with the basic tenets of spatial verification. The framework will improve forecasts by allowing a user to tune the model parameters to have desirable spatial features. The PIs have contributed to the development of the spatial techniques as well as to the sensitivity analysis methods suitable for this problem.Broader Impacts: The broader impacts of the research are multi-pronged. The student involved in this project will become an expert in some of the most sophisticated topics in numerical prediction models: spatial verification, sensitivity analysis, and model tuning. This highly interdisciplinary framework itself has general applicability in that it can be employed to improve predictions from any numerical model involving inexact parameterizations of physical processes or inexact knowledge of physical constants, including weather, climate, ocean, etc. Given that spatial verification is emerging as the verification method of choice for future forecasts, the method will have wide-spread use. The method will be disseminated in the form of journal articles, and computer code to be included in a verification R package produced by National Center for Atmospheric Research (NCAR).