CAREER: Enabling Progressive Data Analytics for High Performance Computing: Algorithms and System Support

点击翻译按钮获取中文摘要

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Rapidly extracting new knowledge from simulation output is critical to the computational sciences at high performance computing (HPC) facilities across the country. However, this has become increasingly challenging due to the growing disparity between the volume of data produced by simulations and the ability to post process the data at the rate it is produced. This project aims to explore reduced representations of data with the overarching goal of achieving science aware and highly adaptable data analytics for HPC applications. The project will create new algorithms and software systems, and benefit the current and future cyberinfrastructure in the U.S. as well as numerous data intensive scientific applications, such as nuclear fusion, astrophysics, combustion, earth science, and others, thus reinforcing the competitiveness and leadership of the United States in this area. Success in the project goals will greatly reduce the time to new knowledge from scientific simulations across various science and engineering disciplines at HPC centers and significantly enhance HPC research and education. The project will contribute to society through engaging underrepresented groups and a set of integrated research and education activities.The project will develop algorithms and system support centered on the idea of leveraging multilevel data representations to enable progressive data analytics on HPC systems. The proposed work fundamentally differs from conventional lossy data compression in that it can guarantee and enforce scientific constraints and augment accuracy based upon applications needs and system state. The project has integrated research and educational activities in algorithms, systems, and applications, taking into account application requirements and architecture trends in large-scale storage to advance the field of scientific data management. More specifically, the project will make contributions in several areas: 1) constraint-based data decomposition; 2) exploiting error-controlled multilevel representations for performance optimization on HPC storage systems; 3) providing a cross-layer solution to mitigate performance variation in containerized environments, with multiprocessor and multi-application coordination achieved through a probabilistic method for selecting the number of levels to retrieve; and 4) integration and evaluation on production science applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.