Collaborative Research: CNS Core: Medium: Optimizing Storage Caches via Adaptive and Reconfigurable Tiering

新型的数据存储和存储设备正在开发和发布，但它们具有非常不同的属性，例如速度，成本，尺寸，可靠性和能量。 有了这些新选项，就有机会降低存储的成本或环境影响，同时提高其可靠性和性能。 为了实现这些好处，该项目探索了将新存储设备一起使用的技术。 通过探索何时以及如何在新存储“层”之间移动数据，以及自动确定在数据存储层中使用什么设备，该项目极大地改善了供应商和最终用户的存储。该项目分析了通过三个相互关联的重点，使用机器学习和时间序列技术检测最佳重新配置点的方法。在第一个推力，准确的，分析，多层尾部延迟模型开发使用排队论。 然后，该项目建立了一个有效的平台来模拟配置和研究方法来估计分层缓存的重新配置成本。 最后，轻量级的，低开销，准确的采样技术，探索运行系统，快速检测显着的输入/输出和缓存行为的变化。 作为回应，该项目进一步构建了在运行系统上以最小干扰重新配置分层缓存的技术。 存储社区从该项目产生的多个工件中受益：开源多功能多层缓存模拟器，工作负载和分析延迟模型，几个案例研究系统，经验评估设备的指标数据库，以及报告意外或违反直觉结果的出版物。 存储消费者（例如，公司）可以在实际购买任何硬件之前模拟许多“假设”场景，以避免供应不足或过度供应。 该项目开发了新的课程模块，包括简短的视频教程。 包括女性和代表性不足的群体成员在内的几名学生接受了研究技术的指导和培训。该项目的工件-软件，源代码，数据集，多层缓存模拟器，跟踪和结果-都包含在我们称为“MTCache：多层缓存”的系统中。 结果将通过同行评审的出版物和arxiv.org进行传播。所有工件将通过项目网站https://www.filesystems.org/mtcache公开。 该项目计划在项目结束后至少维持该场地十年。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

New types of data storage and memory devices are being developed and released, but they have very different properties, such as speeds, costs, sizes, reliability, and energy. With these new options, there is an opportunity to reduce the cost or environmental impact of storage while improving its reliability and performance. To realize these benefits, this project explores techniques to use new storage devices together. By exploring when and how to move data between “tiers” of new storage, as well as automatically determining what devices to use in data storage tiers, the project dramatically improves storage for both providers and the end users.This project analyzes methods to detect optimal reconfiguration points using machine-learning and time-series techniques via three interrelated thrusts. In the first thrust, accurate, analytical, multi-tier tail latency models are developed using queuing theory. Then, the project builds an efficient platform to simulate configurations and investigates methods to estimate tiered-cache reconfiguration costs. Finally, lightweight, low-overhead, accurate sampling techniques are explored for running systems, to quickly detect significant input/output and cache behavior changes. In response, this project further builds techniques to reconfigure a tiered-cache on running systems with minimal interference. Empirical case studies are applied to Memcached and Kubernetes containers.The storage community benefits from multiple artifacts this project produces: an open-source versatile multi-tier cache simulator, workload and analytical latency models, several case studied systems, a database of metrics from empirically evaluated devices, and publications reporting unexpected or counter-intuitive results. Storage consumers (e.g., companies) can simulate many “what-if” scenarios before actually purchasing any hardware, so as to avoid under- or over-provisioning. This project develops new course modules including short video tutorials. Several students, including women and members of underrepresented groups, are mentored and trained in research techniques.The project's artifacts—software, source code, data sets, multi-tier cache simulator, traces, and results—are all embodied in a system we call "MTCache: Multi-Tier Cache". Results will be disseminated using peer-reviewed publications and arxiv.org. All artifacts will be made public through the project Website: https://www.filesystems.org/mtcache. The project plans to maintain that site for at least ten years following the end of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.