OAC Core: LABIOS: Storage Acceleration via Data Labeling and Asynchronous I/O

点击翻译按钮获取中文摘要

Data is everywhere, and it is growing exponentially. Data-intensive computing is a field that deals with processing and analyzing massive amounts of data for various purposes, such as scientific discovery, business intelligence, social media, and health care. However, storing and accessing data efficiently and effectively is a major challenge for data-intensive computing. Current storage systems are not designed to handle the diverse and complex data needs of different applications. They are slow, rigid, and wasteful. This project introduces a new way of managing data that overcomes these limitations. It develops a new data representation and a companion storage system that are faster, more flexible, and more scalable than existing solutions. This project also shows how this new approach improves the performance and functionality of various applications in different application domains, such as scientific computing, deep learning, and microservices. It advances the state-of-the-art in data-intensive computing and serves the national interest by enhancing scientific productivity and output, increasing the scalability and reliability of scientific applications that handle massive and complex data sets, and facilitating the collaboration and communication among scientists from different disciplines by improving the compatibility of scientific applications that use different data formats, platforms, and protocols. This project can foster interdisciplinary research and innovation. It also supports education and diversity in computing and data science by creating a well-trained workforce and enhancing the existing curricula and research programs.The goal of this project is to develop and evaluate a new input/output (I/O) paradigm for data-intensive computing based on the concept of a Label and a system called LABIOS. A Label is a data unit that encapsulates an operation and a pointer to its input data. LABIOS is a distributed storage system that executes Labels asynchronously and independently on servers that support tiered storage (e.g., non-volatile memory express (NVMe), solid state drive (SSD), hard disk drive (HDD), etc.). This project investigates how to design and implement LABIOS to provide active storage semantics, software defined storage semantics, and in-situ and in-transit analytics. It explores how to use LABIOS to support elastic and dynamic resource provisioning, integrated storage access, and programmable storage application programming interfaces (APIs). The performance and functionality of LABIOS are evaluated using various applications and workloads from scientific and Big Data domains, such as scientific computing, deep learning, and microservices; and how LABIOS improves the I/O efficiency and effectiveness of these applications and workloads are demonstrated. This project also analyzes the implications of the new I/O paradigm for the existing high-performance computing (HPC) ecosystem. It contributes to the advancement of data-intensive computing by providing a novel data representation and a scalable storage system that can address the diverse and complex data requirements of large-scale applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.