Statistical Analysis of Massive Spatio-Temporal Datasets Using Distributed Computing

卫星和飞机上的自动传感仪器使人们能够收集大量的空间和时间索引数据。如果这类数据集能够得到有效利用，它们可以为各种问题提供新的见解，例如气候的温室气体浓度，精准农业的土壤特性以及天气预报的大气状态。然而，传统的空间统计技术在计算上不适用于大数据集。该项目将开发快速和用户友好的软件，可以填补空白，捕捉从非常精细到非常大的尺度的不均匀空间结构，并适当量化不确定性。例如，该软件将应用于对每小时总可降水量场的数百万次卫星测量，这对恶劣天气预报至关重要，该项目的目标是开发对大规模高分辨率空间数据集进行统计分析的方法。所采用的模型是指定在多个分辨率的空间基函数。选择基函数以最佳地近似给定的协方差函数。不需要对协方差函数进行任何限制，并且观测的间隔可以是不规则的。至关重要的是，基函数表示的结构导致可扩展的并行推理算法，可以充分利用现代计算环境中可用的许多节点。这一方法将扩展到时空数据分析，从而能够对海量流时空数据集进行实时分析。所有方法都将在适用于多核台式计算机和超级计算环境的软件中实现。

Automated sensing instruments on satellites and aircraft have enabled the collection of massive amounts of data indexed in space and time. If these kinds of datasets can be efficiently exploited, they can provide new insights on a wide variety of issues, such as greenhouse gas concentrations for climate, soil properties for precision agriculture, and atmospheric states for weather forecasting. However, traditional spatial-statistical techniques are not computationally feasible for big datasets. This project will develop fast and user-friendly software that can fill gaps, capture inhomogeneous spatial structure from very fine to very large scales, and properly quantify uncertainty. As an illustration, the software will be applied to millions of satellite measurements of hourly Total Precipitable Water fields, which are critical in severe weather forecasting.The goal of this project is to develop methodology for the statistical analysis of massive, high-resolution spatial datasets. The employed model is specified in terms of spatial basis functions at multiple resolutions. The basis functions are chosen to optimally approximate a given covariance function. No restrictions on the covariance function are necessary, and observations can be irregularly spaced. It is crucial that the structure of the basis-function representation results in scalable, parallel inference algorithms that can take full advantage of the many nodes available in modern computing environments. The methodology will be extended to the analysis of spatio-temporal data, allowing real-time analysis of massive, streaming spatio-temporal datasets. All methods will be implemented in software suitable for both multi-core desktop computers and supercomputing environments.