CSR: Small: Collaborative Research: Adaptive Memory Resource Management in a Data Center - A Transfer Learning Approach

云计算已经成为在线服务和传统数据密集型计算的主要可扩展计算平台（示例包括Amazon的EC2、Microsoft的Azure、IBM的SmartCloud等）。 云计算数据中心在大量用户之间共享计算资源，提供了一种成本有效的手段来允许用户访问对个人来说不切实际的计算能力和数据存储。数据中心经常不得不过度使用其资源来满足服务质量合同。数据中心软件需要有效地管理其资源，以满足用户提交各种应用程序的需求，而无需事先了解这些应用程序。这项工作的重点是在数据中心的内存资源的管理问题。迁移学习方法的最新进展启发了这项工作，即创建动态模型来预测应用程序的该高速缓存和内存需求。该项目有四项主要任务：（i）对迁移学习的最新进展如何帮助解决数据中心资源管理问题的调查，（ii）使用运行中虚拟机测量来开发动态高速缓存预测器，（iii）使用虚拟机的运行时特性来创建动态存储器预测器，以及（iv）开发统一的资源管理方案，创建一组动态调整高速缓存和内存分配以实现服务质量目标的策略。 在任务（i）-（iii）中，采用并探索迁移学习方法，以促进基于现有系统和基准应用程序的广泛培训的知识和模型向新系统环境和应用程序的迁移。 预测模型和管理方案将在SPEC WEB和CloudSuite 2.0等通用基准上进行评估。 本研究的结果将对云计算数据中心的设计和实现产生广泛的影响。研究结果将有助于提高云计算虚拟化系统的资源利用率、系统吞吐量和预测性能。此外，所设计的方法和它们所传授的知识将促进对系统研究和机器学习的理解。

Cloud computing has become a dominant scalable computing platform for both online services and conventional data-intensive computing (examples include Amazon's EC2, Microsoft's Azure, IBM's SmartCloud, etc.). Cloud computing data centers share computing resources among a large set of users, providing a cost effective means to allow users access to computational power and data storage not practical for an individual. A data center often has to over-commit its resources to meet Quality of Service contracts. The data center software needs to effectively manage its resources to meet the demands of users submitting a variety of applications, without any prior knowledge of these applications. This work is focused on the issue of management of memory resources in a data center. Recent progress in transfer learning methods inspires this work in the creation of dynamic models to predict the cache and memory requirements of an application. The project has four main tasks: (i) an investigation into how recent advancements in transfer learning can help solve data center resource management problems, (ii) development of a dynamic cache predictor using on-the-fly virtual machine measurements, (iii) creation of a dynamic memory predictor using runtime characteristics of a virtual machine, and (iv) development of a unified resource management scheme creating a set of heuristics that dynamically adjust cache and memory allocation to fulfill Quality of Service goals. In tasks (i)-(iii), transfer learning methods are employed and explored to facilitate the transfer of knowledge and models to new system environments and applications based on extensive training on existing systems and benchmark applications. The prediction models and management scheme will be evaluated on common benchmarks including SPEC WEB and CloudSuite 2.0. The results of this research will have broad impact on the design and implementation of cloud computing data centers. The results will help improve resource utilization, boost system throughput, and improve predication performance in a cloud computing virtualization system. Additionally, the methods designed and knowledge they impart will advance understanding in both systems research and machine learning.