CSR: Small: A Unified Reinforcement Learning Approach for Autoconfiguration of Virtualized Resources and Appliances

该奖项是根据2009年美国复苏和再投资法案（公法111-5）资助的。通过虚拟化解锁的云计算正在成为越来越重要的面向服务的计算范式。该项目的目标是开发一种统一的学习方法，即URL，以自动化虚拟机和运行在虚拟机上的应用程序的配置过程，并使系统配置适应云的动态。URL方法有三个创新：首先是一种用于在分布式计算资源上实时自动配置虚拟机的强化学习（RL）方法。第二个是统一的RL方法，用于自动配置vm和多层web设备。它能够以协调的方式调整VM资源预算和设备参数设置，以适应云动态和不断变化的工作负载，以实现服务质量保证的目标。第三种是分布式、协作式强化学习方法，它允许基于强化学习的学习和优化代理在不同的服务器上运行，并具有独立的操作选择，从而在大规模系统中制定最佳的联合配置策略。从这个项目中产生的可交付成果将促进对大规模复杂系统的自主管理的发现和理解，具有深远的技术、经济和社会影响。此外，这个项目有一个完整的教育组成部分。它将提高对系统管理问题和机器学习技术力量的认识水平，并为学生进入云计算行业做好准备，充分了解云计算的挑战和机遇。

URL: A Unified Reinforcement Learning Approach for Auto-configuration ofVirtualized Resources and AppliancesThis award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).Cloud computing, unlocked by virtualization, is emerging as an increasingly important service-oriented computing paradigm. The goal of this project is to develop a unified learning approach, namely URL, to automate the configuration processes of virtualized machines and applications running on the virtual machines and adapt the systems configuration to the dynamics of cloud. The URL approach features three innovations: First is a reinforcement learning (RL) methodology for auto-configuration of virtual machines (VMs) on distributed computing resources in a real-time manner. Second is a unified RL approach for auto-configuration of both VMs and multi-tier web appliances. It is able to adapt the VM resource budget and appliance parameter settings in a coordinated way to the cloud dynamics and the changing workload for the objective of service quality assurance. Third is a distributed, cooperative RL approach that allows the RL-based learning and optimization agents running on different servers and with independent action choices to make an optimal joint configuration policy in large-scale systems. Deliverables that emerge from this project will advance discovery and understanding of autonomic management of large-scale complex systems with profound technical, economic, and societal impact. In addition, this project has an integral educational component. It will raise the level of awareness of system management issues and the power of machine learning technology, and prepare the students to enter the industry with adequate understanding of the challenges and opportunities in cloud computing.