ITR-(ASE)-(dmc+int): Reconfigurable, Data-driven Resource Allocation in Complex Systems: Practice and Theoretical Foundations

复杂系统中可重新配置的数据驱动的资源分配：实践和理论基础随着Web服务器发展成为我们社会信息基础设施的核心组件，在以经济高效的方式确保高可用性的同时，为其不断增长和多样化的客户群提供服务变得具有挑战性。今天Web服务器系统的复杂性及其工作负载的多变性往往使有效的资源分配成为一个难以实现的目标。该提案寻求支持开发一个数据驱动的性能工程框架，以在当今复杂的Web服务器系统中自动执行健壮的、工作负载感知的资源分配和管理过程。研究人员的重点是更好地了解工作量资源需求，制定和实施有效的方法，以便在系统一级查明瓶颈和分配资源，以及制定有效的分析方法进行业绩预测。为了实现上述目标，将完成以下研究任务：o将更好地了解提供动态页面服务的Web服务器中的工作负载资源需求，重点是确定不同的资源瓶颈和触发这些瓶颈的工作负载条件。o将在系统一级设计一个数据收集机制，收集统计信息，这可以促使调度器重新配置。这一机制将更好地理解需要在运行时监视什么系统和工作负载数据，以及在什么详细程度上，以便容易地向分配策略提供关于系统状态的信息。o将为各种瓶颈资源开发新的数据驱动的调度策略，并将在系统级别实施该策略，以允许在瞬时过载条件下快速系统恢复。o新的理论结果将允许使用紧凑和易处理的模型对工作负载和资源分配策略进行建模。这些模型将指导资源分配策略的参数化。智能优点：拟议的研究将通过集成数据和分析模型来推动科学和工程，以便在实际系统上开发和实施工作负载感知和系统感知算法，以调节在不断变化的工作负载条件下提供动态页面的Web服务器中的资源分配。这项拟议的研究，即使假设不是所有的结果都是积极的，也将试图回答开发具有成本效益的自主系统的几个基本问题。这项研究的理论贡献将推动复杂系统建模方面的最先进水平，这些系统受到工作负载强度和需求的持续和严重变化的影响。广泛影响：本研究的影响将通过行业合作，特别是希捷研究，通过提供可以调节和自动化复杂环境中的资源分配过程的算法和工具，影响实际现成系统中的实践状态。通过这个项目，研究人员还将能够影响几个学生的教育，使他们做好准备，更好地满足复杂环境下性能建模和资源分配领域的行业需求。

Reconfigurable, data-driven resource allocation in complex systems:practice and theoretical foundationsAs web servers are developing into central components in the information infrastructure of our society, it becomes challenging to serve their ever-increasing and diversified customer population while ensuring high availability in a cost-effective way. The complexity of today's web servers systems and the variability of their workload often make effective resource allocation an elusive goal.This proposal seeks support for the development of a data-driven performance-engineering framework to automate the process of robust, workload-aware resource allocation and management in today's complex web server systems. The researcher's focus is on the development of better understanding of the workload resource demands, on the development and implementation of efficient methodologies for bottleneck identification and resource allocation at the system level, and on the development of efficient analytic methodologies for performance prediction. To meet the above targets the following research tasks will be accomplished:o A better understanding of the workload resource demands in web servers that serve dynamic pages will be obtained, focusing on identifying the different resource bottlenecks and the workload conditions under which these bottlenecks are triggered.o A data collection mechanism at the system level will be devised that will gather statistical information, which can prompt scheduler reconfigurations. This mechanism will provide a better understanding on what system and workload data, and at what level of detail, needs to be monitored at run-time to readily provide to the allocation policies information about the state of the system.o New, data-driven scheduling policies will be developed and will be implemented at the system level for the various bottleneck resources that will allow quick system recovery under transient overload conditions.o New theoretical results will allow modeling of the workload and resource allocation policies with compact and tractable models. These models will guide parameterization of the resource allocation policies.Intellectual Merit: The proposed research will advance science and engineering by integrating data and analytic models for the development and implementation on actual systems of both workload-aware and system-aware algorithms to modulate resource allocation in web servers serving dynamic pages under constantly changing workload conditions. The proposed research, even assuming that not all results are positive, will attempt to answer several fundamental questions for the development of cost-effective, autonomic systems. The theoretical contributions of this research will advance the state-of-the-art in modeling of complex systems that are subject to continuous and severe changes in workload intensities and demands.Broader Impact: The impact of this research will affect that state-of-the-practice in actual off-the-shelf systems via industrial collaborations, specifically Seagate Research, by providing algorithms and tools that can modulate and automate the process of resource allocation in complex environments. Through this project, the researcher will also be able to impact the education of several students, preparing them to better meet industry demands in the areas of performance modeling and resource allocation in complex environments.