Evaluating the Potential and Challenges of an Uncertainty Quantification Method for Long Short‐Term Memory Models for Soil Moisture Predictions
Evaluating the Potential and Challenges of an Uncertainty Quantification Method for Long Short‐Term Memory Models for Soil Moisture Predictions
复制标题
DOI：
10.1029/2020wr028095
复制
发表时间：
2020-06
影响因子：
5.4
通讯作者：
K. Fang;Daniel Kifer;K. Lawson;Chaopeng Shen
中科院分区：
文献类型：
--
作者：
K. Fang;Daniel Kifer;K. Lawson;Chaopeng Shen
最近，经常性的深层网络已经显示出利用新的卫星遥感数据进行长期土壤水分预测的前景。然而，为了在预测中发挥作用，深度网络还必须提供不确定性估计。在这里，我们使用输入相关数据噪声项（MCD+N）评估了Monte Carlo dropout，这是一种最初在计算机视觉中开发的有效不确定性估计框架，用于水文时间序列预测。MCD+N同时估计异方差输入相关数据噪声项（归因于观测噪声的训练误差模型）和网络权重不确定性项（归因于约束不足的模型参数）。虽然MCD+N有吸引力的功能，许多启发式的近似在其推导过程中，并缺乏严格的评估和证据，其断言的能力，以检测相异性。为了解决这个问题，我们对该计划的潜力和局限性进行了深入评估。我们发现，为了再现土壤水分主动被动（SMAP）任务（使命）记录的土壤水分动态，MCD+N确实给出了很好的预测误差估计，前提是我们调整了超参数并使用了代表性的训练数据集。依赖于输入的项对观测噪声有很强的响应，而模型项显然充当了训练数据的地文差异检测器，表现如预期。然而，当训练和测试数据在特征上不同时，输入依赖项可能会被误导，从而破坏其可靠性。此外，由于模型的数据驱动性质，数据噪声也会影响网络权重的不确定性，因此这两个不确定性项是相关的。总的来说，这种方法是有希望的，但需要小心解释结果。
Recently, recurrent deep networks have shown promise to harness newly available satellite‐sensed data for long‐term soil moisture projections. However, to be useful in forecasting, deep networks must also provide uncertainty estimates. Here we evaluated Monte Carlo dropout with an input‐dependent data noise term (MCD+N), an efficient uncertainty estimation framework originally developed in computer vision, for hydrologic time series predictions. MCD+N simultaneously estimates a heteroscedastic input‐dependent data noise term (a trained error model attributable to observational noise) and a network weight uncertainty term (attributable to insufficiently constrained model parameters). Although MCD+N has appealing features, many heuristic approximations were employed during its derivation, and rigorous evaluations and evidence of its asserted capability to detect dissimilarity were lacking. To address this, we provided an in‐depth evaluation of the scheme's potential and limitations. We showed that for reproducing soil moisture dynamics recorded by the Soil Moisture Active Passive (SMAP) mission, MCD+N indeed gave a good estimate of predictive error, provided that we tuned a hyperparameter and used a representative training data set. The input‐dependent term responded strongly to observational noise, while the model term clearly acted as a detector for physiographic dissimilarity from the training data, behaving as intended. However, when the training and test data were characteristically different, the input‐dependent term could be misled, undermining its reliability. Additionally, due to the data‐driven nature of the model, data noise also influences network weight uncertainty, and therefore the two uncertainty terms are correlated. Overall, this approach has promise, but care is needed to interpret the results.