Hardware-augmented database engines in a dark silicon era

点击翻译按钮获取中文摘要

Database technology forms a critical underpinning of modern infrastructure, supporting key fields such as banking, commerce, internet services, governments and travel. Database management systems (DBMS) are successful because they provide a stable interface for applications to target, while continually reinventing themselves to exploit new hardware capabilities and tame a growing deluge of data. Unfortunately, a looming hardware crisis, known as "dark silicon," threatens our ability to control the digital fire hose. Power constraints and limitations in achievable parallelism are forcing systems to idle an increasing fraction of their compute resources at any given time; increasingly, application performance will plateau unless we can reduce the energy that computations consume. Such reductions come from speeding up a computation (so tasks finish sooner) or by improving power efficiency (so more tasks can run at once). Clever software techniques can help, but general purpose processors - by their very generality - impose severe limits on the gains software can achieve. Dark silicon thus favors a variety of specialized hardware units capable of operating at vastly higher efficiencies than general purpose processors; units not useful to a particular computation are disabled in favor of other, more useful ones. As we enter the dark silicon era, the continued success of the DBMS will thus hinge on its ability to utilize specialized hardware. Unlike many areas of high-performance computing, however, database engines are quite resistant to hardware acceleration. The database engine must again reinvent itself, this time to become more hardware-friendly. This project aims to re-examine database engines from the ground up, redesigning the most expensive and time consuming database operations around tailor-made hardware support; similar strategies have already yielded significant advances in areas such as operating systems and virtual machines. The resulting hardware-software symbiosis will endow the DBMS with the unprecedented efficiency and performance it needs to collect and process data in a power-constrained world.