Hardware-augmented database engines in a dark silicon era

数据库技术构成了现代基础设施的关键基础设施，支持银行、商业、互联网服务、政府和旅游等关键领域。数据库管理系统(DBMS)之所以成功，是因为它们为应用程序提供了一个稳定的接口，同时不断地自我改造，以利用新的硬件功能并驯服日益增长的数据洪流。不幸的是，一场迫在眉睫的硬件危机，也就是所谓的“暗硅”，威胁着我们控制数字消防水龙的能力。电源限制和可实现的并行性限制正迫使系统在任何给定时间闲置越来越多的计算资源；除非我们能够减少计算消耗的能量，否则应用程序性能将日益停滞。这种减少来自于加速计算(以便更快地完成任务)或通过提高能效(以便可以同时运行更多任务)。聪明的软件技术可以有所帮助，但通用处理器--由于其非常普遍--对软件所能实现的收益施加了严格的限制。因此，暗硅有利于能够以比通用处理器高得多的效率运行的各种专用硬件单元；对特定计算没有用处的单元被禁用，以支持其他更有用的单元。随着我们进入黑暗的硅时代，数据库管理系统的持续成功将取决于它利用专门硬件的能力。然而，与许多高性能计算领域不同的是，数据库引擎对硬件加速的抵抗力很强。数据库引擎必须再次自我改造，这一次要变得更加硬件友好。该项目旨在从头开始重新检查数据库引擎，围绕定制的硬件支持重新设计最昂贵和最耗时的数据库操作；类似的策略已经在操作系统和虚拟机等领域取得了重大进展。由此产生的软硬件共生将赋予数据库管理系统前所未有的效率和性能，使其能够在电力有限的世界中收集和处理数据。

Database technology forms a critical underpinning of modern infrastructure, supporting key fields such as banking, commerce, internet services, governments and travel. Database management systems (DBMS) are successful because they provide a stable interface for applications to target, while continually reinventing themselves to exploit new hardware capabilities and tame a growing deluge of data. Unfortunately, a looming hardware crisis, known as "dark silicon," threatens our ability to control the digital fire hose. Power constraints and limitations in achievable parallelism are forcing systems to idle an increasing fraction of their compute resources at any given time; increasingly, application performance will plateau unless we can reduce the energy that computations consume. Such reductions come from speeding up a computation (so tasks finish sooner) or by improving power efficiency (so more tasks can run at once). Clever software techniques can help, but general purpose processors - by their very generality - impose severe limits on the gains software can achieve. Dark silicon thus favors a variety of specialized hardware units capable of operating at vastly higher efficiencies than general purpose processors; units not useful to a particular computation are disabled in favor of other, more useful ones. As we enter the dark silicon era, the continued success of the DBMS will thus hinge on its ability to utilize specialized hardware. Unlike many areas of high-performance computing, however, database engines are quite resistant to hardware acceleration. The database engine must again reinvent itself, this time to become more hardware-friendly. This project aims to re-examine database engines from the ground up, redesigning the most expensive and time consuming database operations around tailor-made hardware support; similar strategies have already yielded significant advances in areas such as operating systems and virtual machines. The resulting hardware-software symbiosis will endow the DBMS with the unprecedented efficiency and performance it needs to collect and process data in a power-constrained world.