Collaborative Research: CNS Core: Medium: Optimizing Storage Caches via Adaptive and Reconfigurable Tiering

新型数据存储和存储设备正在开发和发布，但它们具有非常不同的特性，如速度、成本、大小、可靠性和能量。有了这些新选项，就有机会降低存储的成本或对环境的影响，同时提高其可靠性和性能。为了实现这些优势，该项目探索了同时使用新存储设备的技术。通过探索何时以及如何在新存储的“层”之间移动数据，以及自动确定在数据存储层中使用哪些设备，该项目极大地改善了提供商和最终用户的存储。该项目分析了通过三个相互关联的推力使用机器学习和时间序列技术来检测最佳重新配置点的方法。在第一个推力中，利用排队论建立了精确的、解析的、多层尾部延迟模型。然后，该项目构建了一个高效的平台来模拟配置，并研究了估算分层缓存重新配置成本的方法。最后，为运行中的系统探索轻量级、低开销、准确的采样技术，以快速检测重要的输入/输出和缓存行为更改。作为回应，该项目进一步构建了在运行的系统上重新配置分层缓存的技术，并将干扰降至最低。存储社区受益于该项目产生的多个构件：开源的多功能多层缓存模拟器、工作负载和分析延迟模型、几个案例研究系统、来自经验性评估设备的指标数据库以及报告意外或违反直觉的结果的出版物。存储消费者(例如，公司)可以在实际购买任何硬件之前模拟许多假设场景，以避免供应不足或供应过度。该项目开发了包括短视频教程在内的新课程模块。一些学生，包括女性和代表不足的群体的成员，接受了研究技术方面的指导和培训。项目的人工制品-软件、源代码、数据集、多层缓存模拟器、跟踪和结果-都体现在一个我们称为“MTCache：多层缓存”的系统中。结果将通过同行评议的出版物和arxiv.org传播。所有文物将通过项目网站https://www.filesystems.org/mtcache.公开。该项目计划在项目结束后对该场地进行至少十年的维护。这一奖励反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

New types of data storage and memory devices are being developed and released, but they have very different properties, such as speeds, costs, sizes, reliability, and energy. With these new options, there is an opportunity to reduce the cost or environmental impact of storage while improving its reliability and performance. To realize these benefits, this project explores techniques to use new storage devices together. By exploring when and how to move data between “tiers” of new storage, as well as automatically determining what devices to use in data storage tiers, the project dramatically improves storage for both providers and the end users.This project analyzes methods to detect optimal reconfiguration points using machine-learning and time-series techniques via three interrelated thrusts. In the first thrust, accurate, analytical, multi-tier tail latency models are developed using queuing theory. Then, the project builds an efficient platform to simulate configurations and investigates methods to estimate tiered-cache reconfiguration costs. Finally, lightweight, low-overhead, accurate sampling techniques are explored for running systems, to quickly detect significant input/output and cache behavior changes. In response, this project further builds techniques to reconfigure a tiered-cache on running systems with minimal interference. Empirical case studies are applied to Memcached and Kubernetes containers.The storage community benefits from multiple artifacts this project produces: an open-source versatile multi-tier cache simulator, workload and analytical latency models, several case studied systems, a database of metrics from empirically evaluated devices, and publications reporting unexpected or counter-intuitive results. Storage consumers (e.g., companies) can simulate many “what-if” scenarios before actually purchasing any hardware, so as to avoid under- or over-provisioning. This project develops new course modules including short video tutorials. Several students, including women and members of underrepresented groups, are mentored and trained in research techniques.The project's artifacts—software, source code, data sets, multi-tier cache simulator, traces, and results—are all embodied in a system we call "MTCache: Multi-Tier Cache". Results will be disseminated using peer-reviewed publications and arxiv.org. All artifacts will be made public through the project Website: https://www.filesystems.org/mtcache. The project plans to maintain that site for at least ten years following the end of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.