Retro: Targeted Resource Management in Multi-tenant Distributed Systems
Retro: Targeted Resource Management in Multi-tenant Distributed Systems
复制标题
DOI：
--
复制
发表时间：
2015-05
期刊：
影响因子：
--
通讯作者：
Jonathan Mace;P. Bodík;Rodrigo Fonseca;Madan Musuvathi
中科院分区：
文献类型：
--
作者：
Jonathan Mace;P. Bodík;Rodrigo Fonseca;Madan Musuvathi
在由多个租户共享的分布式系统中，有效的资源管理是提供服务质量保证的重要前提。目前部署的许多系统缺乏性能隔离，并且会遇到争用、速度降低甚至停机，这些都是由激进的工作负载或不正确的维护任务（如数据复制）引起的。在这项工作中，我们提出了复古，共享分布式系统的资源管理框架。Retro监视分布式系统内和分布式系统之间的每个租户的资源使用情况，并通过高级API将此信息公开给集中式资源管理策略。策略可以使用Retro的控制点来塑造租户消耗的资源，这些控制点强制执行共享和速率限制决策。我们通过三种策略展示了Retro，为高优先级租户提供瓶颈资源公平性，主导资源公平性和延迟保证，并在五个分布式系统上评估系统：HBase，Yarn，MapReduce，HDFS和Zookeeper。我们的评估表明，Retro具有较低的开销，并实现了策略的目标，准确地检测竞争资源，节流租户负责减速和过载，公平地分配剩余的集群容量。
In distributed systems shared by multiple tenants, effective resource management is an important pre-requisite to providing quality of service guarantees. Many systems deployed today lack performance isolation and experience contention, slowdown, and even outages caused by aggressive workloads or by improperly throttled maintenance tasks such as data replication. In this work we present Retro, a resource management framework for shared distributed systems. Retro monitors per-tenant resource usage both within and across distributed systems, and exposes this information to centralized resource management policies through a high-level API. A policy can shape the resources consumed by a tenant using Retro's control points, which enforce sharing and rate-limiting decisions. We demonstrate Retro through three policies providing bottleneck resource fairness, dominant resource fairness, and latency guarantees to high-priority tenants, and evaluate the system across five distributed systems: HBase, Yarn, MapReduce, HDFS, and Zookeeper. Our evaluation shows that Retro has low overhead, and achieves the policies' goals, accurately detecting contended resources, throttling tenants responsible for slowdown and overload, and fairly distributing the remaining cluster capacity.