A data-driven approach to improving data center efficiency

大规模的数据处理和存储已经成为我们现代社会的基础。几乎所有行业部门都依赖于高效的数据管理和存储，作为推动其业务发展的引擎，最终用户越来越依赖于大规模处理和存储信息所提供的服务。因此，将数量惊人的资源（包括货币资源和环境资源）用于数据的存储和管理可能并不奇怪。例如，最近的一项研究得出结论，到2025年，管理数据海啸可能会消耗全球五分之一的电力。因此，就未来的经济成功和环境影响而言，优化利用系统来处理和存储大规模数据至关重要。

Large-scale processing and storage of data has become the underpinning of our modern society. Nearly all industry sectors depend on efficient data management and storage as the engine that moves their business and end-users have grown dependent on services enabled by large-scale processing and storing of information. Consequently, it is maybe not surprising that a staggering amount of resources (both monetary as well as environmental) are dedicated to the storage and management of data. For example, a recent study concludes that managing the Tsunami of data' could consume one fifth of global electricity by 2025. As a result, making optimal use of systems for processing and storing data at large scale is critical both in terms of future economic success and environmental impact. Of course, the problem of efficiency in compute clusters and data centers is not new. Much research in recent years has focused on many different facets of the problem, too many in fact to include an exhaustive survey in this proposal. Some examples include more energy-efficient hardware and energy-proportional servers, optimizations to the data center cooling system to improve airflow, increasing data center temperatures, new storage media, flexible and scalable schedulers, disaggregated storage to allow separate scaling of storage and compute resources, more light-weight virtualization mechanisms, continuous improvements in the efficiency of ML and AI algorithms and many more. The key observation motivating this proposal is that even today we are still making terribly inefficient use of data center and cluster resources. The advent of server virtualization and hyper-scale data centers, which are optimized for high density, and the rich academic literature on the topic create the impression that today's compute infrastructure is generally efficiently utilized. In contrast, the PI's recent discussions with several large companies operating their own data centers revealed that data centers are still greatly under-utilized, with typical utilization levels in the 5-25% range, even for hyper-scale data centers. For smaller systems utilization is believed to be even worse. The long-term goal of the work in this proposal is to identify, characterize and remove major sources of inefficiency in today's storage and compute infrastructures, and increase their utilization. The methodology will follow the PI's signature approach to research: form collaborations with industry to gain access to real data from production machines, analyze said data to characterize and identify problems, and use rigorous statistical and algorithmic methods to tackle core problems. The proposal outlines a set of specific near-term objectives that the PI identified in her recent work and plans to address over a 5-year horizon. She expects the results will open up new threads of research that will inform a longer term research agenda.