EarthCube IA: Collaborative Proposal: Optimal Data Layout for Scalable Geophysical Analysis in a Data-intensive Environment

EarthCube IA：协作提案：数据密集型环境中可扩展地球物理分析的最佳数据布局

• 批准号: 1541043
• 负责人: Hongfeng Yu
• 金额: $ 33.29万
• 依托单位: University of Nebraska-Lincoln
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1541043&HistoricalAwards=false
• 关键词: EarthCube; IA; Collaborative; Proposal; Optimal

Steady advance in remote sensing, satellite imaging, and computing technology has enabled scientists to study geophysical phenomena of unprecedented resolutions and complexity. Earth observation data generated from space-based satellites or ground-based radar and radiometer facilities are typically time-varying, and multivariate, and can take tera- or even peta-bytes of space to preserve and process. The common practice is to choose and transfer subsets of data from multiple data archive servers to local machines and then conduct data analysis tasks. However, this approach becomes increasingly unsustainable with an exponential growth of observation data size. It becomes an increasing severe problem that scientists can gain detailed observation data but lack suitable and scalable analysis capabilities to study the full extent of data. The team will work closely to develop, evaluate,and deploy the computer infrastructure to improve the performance and scalability of geophysical analysis for scientific discovery and education. By making the system available to other researchers, it will facilitate the development of new scalable solutions.Interactive geosciences applications will be used as an effective means to promote students interest in science and engineering studies, and to attract and retain students for geosciences community growth.This research develops new techniques in support of scalable geophysical analysis in a data-intensive environment. The innovation and the basis of our technique approach are to develop an optimal data layout algorithm for indexing and placing massive heterogeneous observation data across distributed devices of a cluster. The new data layout is tailored to the spatial-temporal characteristics of Earth observation data, and can directly account for advanced compute techniques, including non-volatile storage resources and GPU- and Manycore-based computing nodes, and support high-throughput and high-resolution exploration of large-scale data. The long-term goal is to study theory and technology that enable scalable data management and analysis for the geosciences community.

遥感、卫星成像和计算技术的稳步发展使科学家能够以前所未有的分辨率和复杂性研究地球物理现象。从天基卫星或地基雷达和辐射计设施生成的地球观测数据通常是时变和多元的，可能需要数兆字节甚至数千兆字节的空间来保存和处理。通常的做法是从多个数据归档服务器中选择数据子集并将其传输到本地计算机，然后执行数据分析任务。然而，随着观测数据规模的指数级增长，这种方法变得越来越不可持续。科学家可以获得详细的观测数据，但缺乏合适的和可扩展的分析能力来研究数据的全部范围，这成为一个日益严重的问题。该团队将密切合作，开发，评估和部署计算机基础设施，以提高科学发现和教育的地球物理分析的性能和可扩展性。通过将该系统提供给其他研究人员，它将促进新的可扩展的解决方案的开发。交互式地球科学应用程序将被用作一种有效的手段，以提高学生对科学和工程研究的兴趣，并吸引和留住学生的地球科学社区的增长。这项研究开发了新的技术，支持可扩展的地球物理分析在数据密集的环境。我们的技术方法的创新和基础是开发一个最佳的数据布局算法，索引和放置大量的异构观测数据在集群的分布式设备。新的数据布局是针对地球观测数据的时空特征量身定制的，可以直接考虑先进的计算技术，包括非易失性存储资源和基于GPU和Manycore的计算节点，并支持大规模数据的高吞吐量和高分辨率探索。长期目标是研究理论和技术，为地球科学界提供可扩展的数据管理和分析。