CAREER: Machine Learning Driven Cross-Layer Optimizations for Storage

计算机科学的最新进步使数据的指数增长超过了技术的增长。随着越来越多的移动设备、传感器和数据采集系统产生艾字节的信息，分析获得的数据变得不可行，需要新的方法来存储和计算这些海量数据。提高存储系统的性能和效率对于推动科学进步、降低IT系统的成本和能源消耗以及分析大量数据至关重要。为了实现这一目标，作为这笔赠款的一部分，将在硬件、操作系统、数据中心和应用程序一级开发新的方法。启用这些新技术将为社会带来巨大的好处。首先，该项目开发的新方法将提高存储系统的效率和利用率，减少我们世界的碳足迹。其次，提高存储设备的性能可以实现新的应用，例如利用存储和计算密集型基因组的新处理。作为该项目的一部分，将利用机器学习技术开发跨层优化来改进存储系统的硬件和软件堆栈。现有块存储设备的一个主要挑战是其内部状态对软件的透明性。这项工作通过使用全面的数据监控功能以及控制旋钮来扩展存储设备以特定于应用程序的方式来优化设备，从而解决了这一缺点。从这些智能存储设备获得的遥测数据将被用于训练机器学习模型，以针对特定于应用的行为进行优化，并确定不同存储和计算环境的最佳配置。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advancements in computer science have enabled an exponential data growth outpacing technology scaling. As an increasing number of mobile devices, sensors and data acquisition systems is producing exabytes of information, analyzing the obtained data are becoming unfeasible, requiring novel approaches to store and compute on this vast amount of data. Improving the performance and efficiency of storage systems is of paramount importance to enable scientific progress, to improve the cost and energy consumption of IT systems and to enable analysis of large amounts of data. To achieve this goal, as part of this grant, novel approaches on the hardware, operating systems, data-center and application level will be developed. Enabling such new techniques will provide significant benefit for society. First, the new approaches developed in this project will improve efficiency and utilization of storage systems reducing the carbon footprint on our world. Secondly, improving the performance of storage devices enables novel applications such as new treatments leveraging storage and compute intensive genomics.As part of this project, cross layer optimizations will be developed to improve the hardware and software stack of storage systems using machine learning techniques. One main challenge of existing block storage devices is their transparency of internal state to software. This work addresses this shortcoming by extending storage devices with comprehensive data monitoring capabilities as well as with control knobs to optimize devices in an application specific way. The telemetry data obtained from these smart storage devices will be utilized to train machine learning models to optimize for application specific behavior as well as for determining optimal configurations of heterogeneous storage and compute environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.