Collaborative Research: CNS Core: Medium: Optimizing Storage Caches via Adaptive and Reconfigurable Tiering

新型的数据存储和存储设备正在开发和发布，但它们具有非常不同的特性，例如速度、成本、尺寸、可靠性和能源。有了这些新选项，就有机会降低存储的成本或对环境的影响，同时提高其可靠性和性能。为了实现这些好处，本项目探索了将新存储设备一起使用的技术。通过探索何时以及如何在新存储的“层”之间移动数据，以及自动确定在数据存储层中使用什么设备，该项目极大地改善了提供商和最终用户的存储。该项目通过三个相互关联的推力，分析了使用机器学习和时间序列技术检测最佳重构点的方法。在第一个推力中，利用排队论建立了精确的、解析的、多层的尾部延迟模型。然后，该项目建立了一个有效的平台来模拟配置，并研究了估计分层缓存重构成本的方法。最后，为运行中的系统探索轻量级、低开销、精确的采样技术，以快速检测重要的输入/输出和缓存行为变化。作为响应，本项目进一步构建技术，以最小的干扰在运行的系统上重新配置分层缓存。实证案例研究应用于Memcached和Kubernetes容器。存储社区受益于该项目产生的多个工件：开源的多功能多层缓存模拟器、工作负载和分析延迟模型、几个案例研究系统、经验评估设备的指标数据库，以及报告意外或反直觉结果的出版物。存储消费者（例如，公司）可以在实际购买任何硬件之前模拟许多“假设”场景，以避免供应不足或过度供应。这个项目开发了新的课程模块，包括短视频教程。一些学生，包括妇女和代表性不足群体的成员，接受了研究技术方面的指导和培训。该项目的工件——软件、源代码、数据集、多层缓存模拟器、跟踪和结果——都包含在一个我们称之为“MTCache：多层缓存”的系统中。研究结果将通过同行评审的出版物和arxiv.org进行传播。所有文物将通过项目网站：https://www.filesystems.org/mtcache公开。该项目计划在项目结束后至少维持该遗址十年。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

New types of data storage and memory devices are being developed and released, but they have very different properties, such as speeds, costs, sizes, reliability, and energy. With these new options, there is an opportunity to reduce the cost or environmental impact of storage while improving its reliability and performance. To realize these benefits, this project explores techniques to use new storage devices together. By exploring when and how to move data between “tiers” of new storage, as well as automatically determining what devices to use in data storage tiers, the project dramatically improves storage for both providers and the end users.This project analyzes methods to detect optimal reconfiguration points using machine-learning and time-series techniques via three interrelated thrusts. In the first thrust, accurate, analytical, multi-tier tail latency models are developed using queuing theory. Then, the project builds an efficient platform to simulate configurations and investigates methods to estimate tiered-cache reconfiguration costs. Finally, lightweight, low-overhead, accurate sampling techniques are explored for running systems, to quickly detect significant input/output and cache behavior changes. In response, this project further builds techniques to reconfigure a tiered-cache on running systems with minimal interference. Empirical case studies are applied to Memcached and Kubernetes containers.The storage community benefits from multiple artifacts this project produces: an open-source versatile multi-tier cache simulator, workload and analytical latency models, several case studied systems, a database of metrics from empirically evaluated devices, and publications reporting unexpected or counter-intuitive results. Storage consumers (e.g., companies) can simulate many “what-if” scenarios before actually purchasing any hardware, so as to avoid under- or over-provisioning. This project develops new course modules including short video tutorials. Several students, including women and members of underrepresented groups, are mentored and trained in research techniques.The project's artifacts—software, source code, data sets, multi-tier cache simulator, traces, and results—are all embodied in a system we call "MTCache: Multi-Tier Cache". Results will be disseminated using peer-reviewed publications and arxiv.org. All artifacts will be made public through the project Website: https://www.filesystems.org/mtcache. The project plans to maintain that site for at least ten years following the end of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.