Apollo:: An ML-assisted Real-Time Storage Resource Observer
Apollo:: An ML-assisted Real-Time Storage Resource Observer
复制标题
DOI：
10.1145/3431379.3460640
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
N. Rajesh;H. Devarajan;Jaime Cernuda Garcia;Keith Bateman;Luke Logan;Jie Ye;Anthony Kougkas
中科院分区：
文献类型：
--
作者：
N. Rajesh;H. Devarajan;Jaime Cernuda Garcia;Keith Bateman;Luke Logan;Jie Ye;Anthony Kougkas
应用程序和中间件服务（如数据放置引擎、I/O调度和预取引擎）需要对遥测数据进行低延迟访问，以便做出最佳决策。然而，典型的监控服务将其遥测数据存储在数据库中，以便允许应用程序查询它们，从而导致显著的延迟损失。这项工作提出阿波罗：一种低延迟监控服务，旨在为应用程序和中间件库提供对相关遥测数据的直接访问。监视系统可能会产生干扰和开销，从而降低作业资源的原始性能。然而，拥有系统的当前视图可以帮助中间件服务做出更优化的决策，最终提高整体性能。Apollo从一开始就被设计为提供低延迟，使用发布-订阅者发布-订阅语义，以及低开销，使用自适应间隔以改变轮询遥测数据资源和机器学习之间的时间长度，以便预测实际资源轮询之间遥测数据的变化。这项工作还提供了一些称为I/O管理器的高级抽象，可以进一步帮助中间件库和应用程序做出最佳决策。评估表明，Apollo可以实现亚毫秒级的延迟，以获取复杂的见解，内存开销约为57 MB，CPU开销仅比现有的最先进的系统多7%。
Applications and middleware services, such as data placement engines, I/O scheduling, and prefetching engines, require low-latency access to telemetry data in order to make optimal decisions. However, typical monitoring services store their telemetry data in a database in order to allow applications to query them, resulting in significant latency penalties. This work presents Apollo: a low-latency monitoring service that aims to provide applications and middleware libraries with direct access to relational telemetry data. Monitoring the system can create interference and overhead, slowing down raw performance of the resources for the job. However, having a current view of the system can aid middleware services in making more optimal decisions which can ultimately improve the overall performance. Apollo has been designed from the ground up to provide low latency, using Publish-Subscriber Pub-Sub semantics, and low overhead, using adaptive intervals in order to change the length of time between polling the resource for telemetry data and machine learning in order to predict changes to the telemetry data between actual resource polling. This work also provides some high level abstractions called I/O curators, which can further aid middleware libraries and applications to make optimal decisions. Evaluations showcase that Apollo can achieve sub-millisecond latency for acquiring complex insights with a memory overhead of ~57 MB and CPU overhead being only 7% more than existing state-of-the-art systems.