OAC Core: LABIOS: Storage Acceleration via Data Labeling and Asynchronous I/O

数据无处不在，而且还在呈指数级增长。数据密集型计算是一个处理和分析海量数据的领域，用于各种目的，如科学发现、商业智能、社交媒体和医疗保健。然而，高效地存储和访问数据是数据密集型计算的主要挑战。当前的存储系统不能处理不同应用程序的多样化和复杂的数据需求。它们缓慢、僵化、浪费。该项目引入了一种新的数据管理方法，克服了这些限制。它开发了比现有解决方案更快、更灵活、更可扩展的新数据表示和配套存储系统。该项目还展示了这种新方法如何改进不同应用领域的各种应用程序的性能和功能，例如科学计算、深度学习和微服务。它通过提高科学生产力和产出，提高处理海量和复杂数据集的科学应用程序的可扩展性和可靠性，并通过改善使用不同数据格式、平台和协议的科学应用程序的兼容性，促进来自不同学科的科学家之间的合作和交流，促进数据密集型计算的最先进水平，并服务于国家利益。这个项目可以促进跨学科的研究和创新。它还通过建立一支训练有素的劳动力队伍和加强现有的课程和研究计划，支持计算和数据科学方面的教育和多样性。该项目的目标是开发和评估基于标签概念和名为LABIOS的系统的数据密集型计算的新输入/输出(I/O)范例。标签是封装操作和指向其输入数据的指针的数据单元。Labios是在支持分层存储的服务器(例如，非易失性快速存储器(NVMe)、固态驱动器(SSD)、硬盘驱动器(HDD)等)上异步且独立地执行标签的分布式存储系统。本项目研究如何设计和实施LABIOS以提供主动存储语义、软件定义的存储语义以及就地和在途分析。它探讨了如何使用LABIOS来支持弹性和动态资源调配、集成存储访问和可编程存储应用程序编程接口(API)。使用科学和大数据领域的各种应用程序和工作负载(如科学计算、深度学习和微服务)评估了LABIOS的性能和功能；并展示了LABIOS如何提高这些应用程序和工作负载的I/O效率和有效性。该项目还分析了新I/O范例对现有高性能计算(HPC)生态系统的影响。它通过提供新颖的数据表示和可扩展的存储系统来促进数据密集型计算的发展，可以满足大规模应用程序的多样化和复杂的数据需求。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Data is everywhere, and it is growing exponentially. Data-intensive computing is a field that deals with processing and analyzing massive amounts of data for various purposes, such as scientific discovery, business intelligence, social media, and health care. However, storing and accessing data efficiently and effectively is a major challenge for data-intensive computing. Current storage systems are not designed to handle the diverse and complex data needs of different applications. They are slow, rigid, and wasteful. This project introduces a new way of managing data that overcomes these limitations. It develops a new data representation and a companion storage system that are faster, more flexible, and more scalable than existing solutions. This project also shows how this new approach improves the performance and functionality of various applications in different application domains, such as scientific computing, deep learning, and microservices. It advances the state-of-the-art in data-intensive computing and serves the national interest by enhancing scientific productivity and output, increasing the scalability and reliability of scientific applications that handle massive and complex data sets, and facilitating the collaboration and communication among scientists from different disciplines by improving the compatibility of scientific applications that use different data formats, platforms, and protocols. This project can foster interdisciplinary research and innovation. It also supports education and diversity in computing and data science by creating a well-trained workforce and enhancing the existing curricula and research programs.The goal of this project is to develop and evaluate a new input/output (I/O) paradigm for data-intensive computing based on the concept of a Label and a system called LABIOS. A Label is a data unit that encapsulates an operation and a pointer to its input data. LABIOS is a distributed storage system that executes Labels asynchronously and independently on servers that support tiered storage (e.g., non-volatile memory express (NVMe), solid state drive (SSD), hard disk drive (HDD), etc.). This project investigates how to design and implement LABIOS to provide active storage semantics, software defined storage semantics, and in-situ and in-transit analytics. It explores how to use LABIOS to support elastic and dynamic resource provisioning, integrated storage access, and programmable storage application programming interfaces (APIs). The performance and functionality of LABIOS are evaluated using various applications and workloads from scientific and Big Data domains, such as scientific computing, deep learning, and microservices; and how LABIOS improves the I/O efficiency and effectiveness of these applications and workloads are demonstrated. This project also analyzes the implications of the new I/O paradigm for the existing high-performance computing (HPC) ecosystem. It contributes to the advancement of data-intensive computing by providing a novel data representation and a scalable storage system that can address the diverse and complex data requirements of large-scale applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.