EarthCube IA: Collaborative Proposal: Optimal Data Layout for Scalable Geophysical Analysis in a Data-intensive Environment

点击翻译按钮获取中文摘要

Steady advance in remote sensing, satellite imaging, and computing technology has enabled scientists to study geophysical phenomena of unprecedented resolutions and complexity. Earth observation data generated from space-based satellites or ground-based radar and radiometer facilities are typically time-varying, and multivariate, and can take tera- or even peta-bytes of space to preserve and process. The common practice is to choose and transfer subsets of data from multiple data archive servers to local machines and then conduct data analysis tasks. However, this approach becomes increasingly unsustainable with an exponential growth of observation data size. It becomes an increasing severe problem that scientists can gain detailed observation data but lack suitable and scalable analysis capabilities to study the full extent of data. The team will work closely to develop, evaluate,and deploy the computer infrastructure to improve the performance and scalability of geophysical analysis for scientific discovery and education. By making the system available to other researchers, it will facilitate the development of new scalable solutions.Interactive geosciences applications will be used as an effective means to promote students interest in science and engineering studies, and to attract and retain students for geosciences community growth.This research develops new techniques in support of scalable geophysical analysis in a data-intensive environment. The innovation and the basis of our technique approach are to develop an optimal data layout algorithm for indexing and placing massive heterogeneous observation data across distributed devices of a cluster. The new data layout is tailored to the spatial-temporal characteristics of Earth observation data, and can directly account for advanced compute techniques, including non-volatile storage resources and GPU- and Manycore-based computing nodes, and support high-throughput and high-resolution exploration of large-scale data. The long-term goal is to study theory and technology that enable scalable data management and analysis for the geosciences community.