The DOPPLER project: modelling and optimizing workloads for the cloud

随着组织的发展，其数据中心不可避免地变得更大、更复杂。这些中心通常向内部和外部用户提供许多服务;这些服务通常部署在跨计算设备集群的虚拟机（VM）上，这些虚拟机可以是组织的财产（即，“私有云”）或从公共云提供商（如Amazon EC2）租用。在这两种情况下，无论是私有还是公共，管理“云”都需要针对服务请求的类型和大小（即，工作量）。这种虚拟化和资源调配的实施需要仔细考虑。在指定的时间段内，过高估计一组请求的服务所需的基础设施会导致浪费、利用不足和成本增加。同时，低估未来的工作量也是不可接受的，因为它最终会降低服务质量，并可能导致违反与客户的服务级别协议（SLA）。最大限度地降低基础设施成本，同时确保满足SLA，不仅需要最先进的资源分配和负载平衡技术，还需要详细了解云工作负载本身。 这个研究项目称为DOPPLER，将研究a）通过挖掘云中的相关“大数据”来描述工作负载，从而理解和建模云工作负载的方法，以及B）通过为云提供高级工作负载预测来优化云配置和管理。短期预测需要根据最近的历史以及观察到的趋势和季节性对即将到来的工作量进行准确预测，将对动态放置和供应进行调查。此外，还将评价长期预测，以模拟长期趋势，而长期趋势又可用于规划和提供未来资源。最后，将在我们的行业合作伙伴CA Technologies及其客户的云环境中对建模和预测方法进行经验评估。

As organizations grow, their data centres inevitably become larger and more complex. These centres usually provide numerous services to the internal and external users; such services are commonly deployed on virtual machines (VMs) across a cluster of computing devices, which can either be the property of the organization (i.e., a "private cloud") or be rented from a public cloud provider such as Amazon EC2. In both cases, private or public, managing "the cloud" requires appropriate virtualization and cost-effective provisioning of the infrastructure with respect to the type and size of the service requests (i.e., the workload). The implementation of this virtualization and provisioning needs to be carefully considered. Over-estimating the required infrastructure for a set of requested services in a specified period of time leads to waste, under-utilization, and increased costs. At the same time, under-estimation of the future workload is also unacceptable, since it eventually degrades the quality of the services and may lead to violations of in the Service-Level Agreements (SLAs) with the clients. Minimizing the infrastructure costs, while at the same time ensuring that SLAs are satisfied, requires not only state-of-the-art techniques for resource assignment and load balancing but also a detailed understanding of the cloud workload itself. This research project, called DOPPLER, will study approaches for a) understanding and modelling the cloud workloads by mining the relevant "big data" in the cloud to characterize the workloads, and b) optimizing cloud provisioning and management by providing advanced workload forecasts for the cloud. Short-term forecasting, requiring accurate prediction of the coming workload based on the recent history and the observed trend and seasonality, will be investigated for dynamic placement and provisioning. In addition, longer-term predictions will be evaluated for modelling longer-term trends, which in turn can be used for planning and provisioning of future resources. Finally, the modelling and forecasting approaches will be empirically evaluated in the context of the cloud environments of our industry partners: CA Technologies and its clients.