Workload-aware database monitoring and consolidation
Workload-aware database monitoring and consolidation
复制标题
DOI：
10.1145/1989323.1989357
复制
发表时间：
2011-06
期刊：
影响因子：
--
通讯作者：
C. Curino;E. Jones;S. Madden;H. Balakrishnan
中科院分区：
文献类型：
--
作者：
C. Curino;E. Jones;S. Madden;H. Balakrishnan
在大多数企业中，数据库都部署在专用的数据库服务器上。通常，这些服务器在大部分时间都没有得到充分利用。例如，在来自不同组织的近 200 台生产服务器的跟踪中，我们发现平均 CPU 利用率低于 4%。可以利用这些未使用的容量将多个数据库整合到更少的机器上，从而降低硬件和运营成本。虚拟机 (VM) 技术是解决此问题的一种流行方法。然而，正如我们在本文中所演示的，虚拟机无法充分支持数据库整合，因为数据库对硬件资源提出了一组独特且具有挑战性的需求，而这些需求不太适合基于虚拟机的整合所做的假设。相反，我们的数据库整合系统（名为 Kairos）使用新颖的技术来衡量数据库工作负载的硬件要求，并使用模型来预测这些工作负载的综合资源利用率。我们将整合问题形式化为非线性优化程序，旨在最大限度地减少服务器数量并平衡负载，同时实现接近零的性能下降。我们将 Kairos 与虚拟机进行比较，在类似 TPC-C 的基准测试中，吞吐量提高了 12 倍。我们还测试了我们的方法对从 Wikia.com、Wikipedia、Second Life 和 MIT CSAIL 的生产服务器收集的实际数据的有效性，显示绝对合并比率在 5.5:1 到 17:1 之间。
In most enterprises, databases are deployed on dedicated database servers. Often, these servers are underutilized much of the time. For example, in traces from almost 200 production servers from different organizations, we see an average CPU utilization of less than 4%. This unused capacity can be potentially harnessed to consolidate multiple databases on fewer machines, reducing hardware and operational costs. Virtual machine (VM) technology is one popular way to approach this problem. However, as we demonstrate in this paper, VMs fail to adequately support database consolidation, because databases place a unique and challenging set of demands on hardware resources, which are not well-suited to the assumptions made by VM-based consolidation. Instead, our system for database consolidation, named Kairos, uses novel techniques to measure the hardware requirements of database workloads, as well as models to predict the combined resource utilization of those workloads. We formalize the consolidation problem as a non-linear optimization program, aiming to minimize the number of servers and balance load, while achieving near-zero performance degradation. We compare Kairos against virtual machines, showing up to a factor of 12× higher throughput on a TPC-C-like benchmark. We also tested the effectiveness of our approach on real-world data collected from production servers at Wikia.com, Wikipedia, Second Life, and MIT CSAIL, showing absolute consolidation ratios ranging between 5.5:1 and 17:1.