Automated Cloud Provisioning and Management

云计算利用数据中心的海量资源能力，以可扩展且经济高效的方式支持服务应用。近年来，云计算基础设施和技术的成功部署正在改变IT业务的运营方式。然而，当今云环境在规模和复杂性方面的快速增长也引发了对底层管理系统的可扩展性和有效性的严重担忧，这使得云资源调配和资源管理成为一个日益困难的挑战。 为了应对这一挑战，我们的长期目标是设计一个可扩展、高效和可靠的云资源管理框架，并将管理开销降至最低。我们的一般方法包括(1)分析工作负载和资源特征，以及(2)开发利用工作负载和资源特征的资源管理方案。我们的方法是基于这样一种观察，即云资源(例如，物理机和网络设备)和工作负载(作业和应用)通常具有异类但具有特征性的行为。例如，面向用户的应用程序通常具有周期性(例如，每日和季节性)需求模式和短期趋势。类似地，具有类似配置的批处理(例如，MapReduct)作业经常在数据中心例行地执行。因此，通过利用从工作负载特征中获得的知识，有可能设计出在降低能耗的同时获得更高的应用程序性能和资源利用率的知情资源管理方案。具体而言，该项目旨在实现以下4个短期目标： (1)云数据中心的工作负载表征是一个比较新的研究课题，尤其是对MapReduceSpark等近期的大数据应用而言。我们最近的工作表明，资源(例如物理硬件的容量、性能、可靠性和能效)和工作负载(例如到达率、优先级、资源使用、通信模式和运行时间)都存在显著的异构性。目前，缺乏以可伸缩但又准确的方式捕获这种异构性的模型。我们打算应用统计学和机器学习的技术来应对这一挑战。 (2)尽管对调度进行了广泛的研究，但针对大数据分析的资源感知和性能感知(如截止期)调度的研究相当有限。在这种情况下，我们计划利用资源和应用程序配置文件，在考虑公平性、性能和资源效率的情况下做出知情的调度决策。 (3)能源成本最小化是当今云提供商的主要关注点。然而，现有的关于这一主题的工作往往忽略了云数据中心的异构性。我们打算开发自适应的异构性感知的能源管理解决方案，以平衡能源效率和应用程序性能之间的平衡。 (4)我们还打算将我们的管理框架扩展到地理分布的云环境中。在这些设置中，以动态方式联合提供服务组件(例如，服务器、负载均衡器、高速缓存、代理)和网络资源以满足性能要求是一大挑战。我们打算开发考虑需求模式的供应方案，以提高地理分布服务的性能和效率。 该项目的成果将极大地推动云资源调配和管理的最先进水平。这也将对工业云管理系统的发展产生深远的影响。最后，它将为培养高素质的博士和硕士人才提供极好的机会。

Cloud computing harnesses massive resource capacity in data centers to support service applications in a scalable and cost-effective manner. The successful deployment of Cloud computing infrastructures and technologies in recent years is transforming the way IT business operates. However, the rapid growth in scale and complexity of today’s Cloud environments also raises critical concerns regarding the scalability and effectiveness of the underlying management systems, making Cloud provisioning and resource management an increasingly difficult challenge. To address this challenge, our long term objective is to design a scalable, efficient and reliable Cloud resource management framework that incurs minimal management overhead. Our general approach consists in (1) analyzing workload and resource characteristics, and (2) developing resource management schemes that leverage workload and resource characterizations. Our approach is motivated by the observation that Cloud resources (e.g., physical machines and network equipment) and workload (jobs and applications) often have heterogeneous yet characterizable behavior. For example, user-facing applications often have periodic (e.g., daily and seasonal) demand patterns and short-term trends. Similarly, batch (e.g., MapReduce) jobs with similar configurations are often executed in data centers on a routinely basis. Thus, by leveraging the knowledge gained from workload characterizations, it is possible to devise informed resource management schemes that attain higher application performance and resource utilization while reducing energy consumption. Specifically, the project aims at achieving the following 4 short-term objectives: (1) Characterizing workload in Cloud data centers is a relatively new research topic, especially for recent big-data applications such as MapReduce and Spark. Our recent work has shown that there is significant heterogeneity of both resources (e.g. capacity, performance, reliability and energy efficiency of physical hardware) and workloads (e.g. arrival rate, priority, resource usage, communication patterns and running time). Currently there is a lack of models for capturing such heterogeneity in a scalable yet accurate fashion. We intend to apply techniques from statistics and machine learning to address this challenge. (2) Despite extensive research on scheduling, the studies on resource-aware and performance (e.g. deadline)-aware scheduling for big-data analytics are rather limited. In this context, we plan to leverage resource and application profiles to make informed scheduling decisions with consideration to fairness, performance and resource efficiency. (3) Minimizing energy cost is a major concern of today’s Cloud providers. However, existing work on this topic often overlooks the heterogeneity in Cloud data centers. We intend to develop adaptive heterogeneity-aware energy management solutions to balance the trade-off between energy efficiency and application performance. (4) We also intend to extend our management framework to the context of geo-distributed clouds. In these settings, it is a major challenge to jointly provision service components (e.g., servers, load balancers, caches, proxies) and network resources to meet performance requirements in a dynamic manner. We intend to develop provisioning schemes that consider demand patterns to improve the performance and efficiency of geo-distributed services. The outcome of this project will significantly advance the state-of-the-art on Cloud resource provisioning and management. It will also have a profound impact on the development of industrial Cloud management systems. Finally, it will provide excellent opportunities for training highly qualified personal at the PhD and Master's levels.