EAGER: Using Machine Learning to Increase the Operational Efficiency of Large Distributed Systems

点击翻译按钮获取中文摘要

Large, distributed systems are nowadays ubiquitous and part of sustainable IT solutions to a broad range of customers and applications. Data centers in the private or public cloud and high performance computing systems are two examples of complex, highly distributed systems: the former are used by almost everyone on a daily basis, the latter are used by computational scientists for advancing science and engineering. High availability and reliability of these complex systems are important for the quality of user experience. Efficient management of such systems contributes to their availability and reliability, and relies on a priori knowledge of the timing of the collective demands of users and a priori knowledge of certain performance measures (e.g., usage, temperature, power) of various systems components.This project aims to provide a systematic methodology to improve the operational efficiency of complex, distributed systems by developing neural networks that can efficiently and accurately predict the incoming workload within fine and coarse time scales. Such workload prediction can dramatically improve the operational efficiency of data centers and high performance systems by driving proactive management strategies that specifically aim to enhance reliability. For datacenters, the focus is on actively reducing performance tickets that are automatically triggered by pro-actively managing virtual machine resizing and migration. For high performance computing systems the focus is on predicting hardware faults to autonomically improve the scheduler's efficiency, direct cooling, and improve performance and memory bandwidth.