III: Medium:Simplifying Database Management with Automated Experimentation

点击翻译按钮获取中文摘要

Despite a number of recent efforts, current solutions for database administration tasks like tuning, troubleshooting, benchmarking, and capacity-planning remain far from satisfactory. Database systems have many configuration parameters to control memory distribution, I/O optimization, costing of query plans, and other behavior. Regular users and even expert database administrators struggle to tune these parameters for good performance. The inherent complexity makes it hard for these systems to gracefully handle uncertain and dynamically-changing workloads and diverse query mixes. The .eX project addresses these challenges through a novel methodology called automated experiment-driven management which encapsulates key building blocks to automatically generate knowledge of system behavior for simplified administration and self-tuning. These building blocks include techniques to: (i) characterize good system performance models, (ii) plan experiments that generate data to learn and maintain these models efficiently under system and workload changes, and (iii) make robust decisions using these models given the inherent uncertainty in how accurately the models capture the true underlying system behavior. Apart from system administration, .eX's automated experiment-driven management can benefit applications like MapReduce computations, large-scale computational simulations, and keyword auctions in online advertising. Two new courses for graduate and undergraduate students at Duke cover principles of automated experiment-driven management. A fully-functional prototype of .eX is being developed and deployed in multiple settings to simplify database administration. The source code of .eX will be released publicly and the technology will be migrated potentially to industrial-strength system administration products. Results from .eX will be disseminated via the project Web site (http://www.cs.duke.edu/~shivnath/dotex.html).