ITR: Data-Driven Autonomic Performance Modulation for Servers

高性能计算机系统的商品化已经导致其作为服务器在许多环境中的广泛部署。我们发现集群和/或对称多处理器（SMP）被广泛用于商业服务，如电子商务和交易处理，日常文件/Web服务的需求，并在学术/研究环境中长期运行的科学应用。虽然部署这些系统的硬件和软件采购成本已经显著下降，但由于涉及人员管理和调优这些系统的成本，总拥有成本实际上正在增加。如今，针对每个环境/配置调优系统和服务是一项重要的任务。该提案旨在开发一个数据驱动的反馈框架-称为巡航控制-以帮助设计和部署这样的自主服务器。有几个研究问题，以调查在这个框架的发展：什么系统和工作负载事件（数据），我们应该在服务器上监控，其性能的后果？我们如何以有意义的方式表示和存储这些数据（并在此过程中压缩它们），因为它可能是在几天内以非常精细的分辨率收集的？根据这些历史数据和当前不断变化的条件，我们如何设计一个控制器，可以调节服务器执行，以避免性能瓶颈的成本效益的方式？在底层操作系统和中间件层中需要什么系统机制来提供数据收集和服务器调制功能？我们如何在不降低服务器性能的情况下，在商用系统上构建和开发这个完整的基础设施？Cruise Control框架将提供收集数据的系统机制，并将尝试对其进行表征以压缩其表示。并且将在操作系统内开发用于实现这种性能调制的机制也将在操作系统内开发。这个一般的框架将实施和实验验证两种不同的服务器环境-商业数据库服务器和高性能计算服务器的科学应用-在集群和SMP硬件平台。

The commoditization of high performance computer systems has resulted in their widespread deployment as servers in numerous environments. We find clusters and/or Symmetric Multiprocessors (SMPs) being extensively used for commercial services such as e-commerce and transaction processing, everyday file/web service needs, and for long running scientific applications in academic/research settings. While hardware and software procurement costs for their deployment have dropped significantly, the total cost of ownership is in fact growing because of the costs of involving a human in managing and tuning these systems. It is a non-trivial task today to tune a system and service for each environment/configuration. This proposal intends to develop a data-driven feedback framework - called Cruise Control - to aid in the design and deployment of such autonomic servers. There are several research questions to be investigated in the development of this framework: What system and workload events (data) should we monitor in the server that have a consequence on its performance? How do we represent and store this data in a meaningful manner (and compress them in the process) since it may be collected over several days and at very fine resolution? Based on this historical data and currently evolving conditions, how do we design a controller that can modulate the server execution to avoid performance bottlenecks in a cost-effective manner? What system mechanisms are needed within the underlying operating system and middleware layer to provide the data collection and server modulation functionalities? How do we structure and develop this complete infrastructure on commodity systems without degrading the performance of the server?The Cruise Control framework will provide system mechanisms for collecting the data and will attempt to characterize them to compress their representation. and will develop within the operating system mechanisms for effecting such performance modulation will also be developed within the operating system. This general framework will be implemented and validated experimentally for two different server environments - a commercial database server and a high performance computing server for scientific applications - on a cluster and a SMP hardware platform.