CSR: Small: Collaborative Research: Adaptive Memory Resource Management in a Data Center - A Transfer Learning Approach

云计算已经成为在线服务和传统数据密集型计算的主要可扩展计算平台（例如亚马逊的EC2、微软的Azure、IBM的SmartCloud等）。云计算数据中心在一大群用户之间共享计算资源，提供了一种经济有效的方法，允许用户访问个人无法使用的计算能力和数据存储。数据中心通常不得不过度承诺其资源以满足服务质量合同。数据中心软件需要有效地管理其资源，以满足用户提交各种应用的需求，而无需事先了解这些应用。这项工作的重点是数据中心内存资源的管理问题。迁移学习方法的最新进展激发了这项工作，即创建动态模型来预测应用程序的缓存和内存需求。该项目有四个主要任务：(i)研究迁移学习的最新进展如何帮助解决数据中心资源管理问题，（ii）使用动态虚拟机测量开发动态缓存预测器，（iii）使用虚拟机的运行时特征创建动态内存预测器，以及（iv）开发统一资源管理方案，创建一组启发式方法，动态调整缓存和内存分配以实现服务质量目标。在任务(i)-(iii)中，采用并探索迁移学习方法，以促进知识和模型转移到基于现有系统和基准应用程序的广泛培训的新系统环境和应用程序中。预测模型和管理方案将在包括specweb和CloudSuite 2.0在内的通用基准上进行评估。本研究结果将对云计算数据中心的设计和实施产生广泛的影响。研究结果将有助于提高资源利用率，提高系统吞吐量，并改善云计算虚拟化系统中的预测性能。此外，他们所设计的方法和传授的知识将促进对系统研究和机器学习的理解。

Cloud computing has become a dominant scalable computing platform for both online services and conventional data-intensive computing (examples include Amazon's EC2, Microsoft's Azure, IBM's SmartCloud, etc.). Cloud computing data centers share computing resources among a large set of users, providing a cost effective means to allow users access to computational power and data storage not practical for an individual. A data center often has to over-commit its resources to meet Quality of Service contracts. The data center software needs to effectively manage its resources to meet the demands of users submitting a variety of applications, without any prior knowledge of these applications. This work is focused on the issue of management of memory resources in a data center. Recent progress in transfer learning methods inspires this work in the creation of dynamic models to predict the cache and memory requirements of an application. The project has four main tasks: (i) an investigation into how recent advancements in transfer learning can help solve data center resource management problems, (ii) development of a dynamic cache predictor using on-the-fly virtual machine measurements, (iii) creation of a dynamic memory predictor using runtime characteristics of a virtual machine, and (iv) development of a unified resource management scheme creating a set of heuristics that dynamically adjust cache and memory allocation to fulfill Quality of Service goals. In tasks (i)-(iii), transfer learning methods are employed and explored to facilitate the transfer of knowledge and models to new system environments and applications based on extensive training on existing systems and benchmark applications. The prediction models and management scheme will be evaluated on common benchmarks including SPEC WEB and CloudSuite 2.0. The results of this research will have broad impact on the design and implementation of cloud computing data centers. The results will help improve resource utilization, boost system throughput, and improve predication performance in a cloud computing virtualization system. Additionally, the methods designed and knowledge they impart will advance understanding in both systems research and machine learning.