Exploring Potential for Resource Request Right-Sizing via Estimation and Container Migration in Apache Mesos
Exploring Potential for Resource Request Right-Sizing via Estimation and Container Migration in Apache Mesos
复制标题
通过 Apache Mesos 中的估计和容器迁移探索资源请求正确大小的潜力
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Gourav Rattihalli
中科院分区：
文献类型：
--
作者：
Gourav Rattihalli
商业云和学术园区集群都面临着资源利用率低和等待时间长的问题，因为用户提供的作业资源估算往往不准确。不正确的资源估计给整个集群和云管理带来了挑战。分配不足可能会导致应用程序显着减慢或终止。为应用程序过度分配资源会导致队列中待处理任务的等待时间增加、吞吐量降低以及集群利用率不足。对于支付资源分配费用的最终用户来说，错误地估计每个作业所需的资源（CPU、内存等）可能会显着增加运行应用程序的总体成本。此外，对于学术云管理者来说，资源碎片是不可接受的，因为他们需要保持较高的利用率，以最大限度地提高资助者的投资回报。我们解决使用 Apache Mesos 资源管理系统的商业和学术云/集群的资源估计问题。我们的愿景是为 Apache Mesos 提供一个资源管理系统，它可以：（1）动态调整每个应用程序所需的资源大小，从而提高整体利用率； (2) 将容器化作业的迁移纳入 Mesos 集群内。
Both commercial clouds and academic campus clusters suffer from low resource utilization and long wait times as the resource estimates for jobs, provided by users, is often inaccurate. Incorrect resource estimation poses challenges in the overall cluster and cloud management. Under allocation can cause significant slowdown or termination of applications. Over-allocation of resources for applications causes increased wait times for pending tasks in the queue, reduced throughput, and underutilization of the cluster. For end users that pay for resource allocations, incorrect estimation of resources (CPU, Memory, etc.) that are needed for each job can significantly increase the overall cost of running applications. Also, for academic cloud managers, resource fragmentation is unacceptable as they need to keep the utilization high to maximize the return on investment for the funding sponsors. We address the resource estimation problem for commercial and academic clouds/clusters that use the Apache Mesos resource management system. Our vision is a resource management system for Apache Mesos that can: (1) dynamically right-size the resources required for each application, thus improving overall utilization; and (2) incorporate migration of containerized jobs within the Mesos cluster.