Hardware-augmented database engines in a dark silicon era

数据库技术是现代基础设施的重要支柱，支持银行、商业、互联网服务、政府和旅游等关键领域。数据库管理系统（DBMS）之所以成功，是因为它们为应用程序提供了一个稳定的接口，同时不断地进行自我改造，以利用新的硬件功能并驯服不断增长的数据洪流。不幸的是，一场迫在眉睫的硬件危机，被称为“暗硅”，威胁着我们控制数字消防水带的能力。功率约束和可实现并行性的限制正迫使系统在任何给定时间内闲置越来越多的计算资源；除非我们能够降低计算消耗的能量，否则应用程序的性能将逐渐趋于稳定。这种减少来自于计算速度的加快（这样任务就能更快地完成）或能源效率的提高（这样更多的任务就能同时运行）。聪明的软件技术可以提供帮助，但是通用处理器——由于它们的通用性——对软件可以实现的增益施加了严重的限制。因此，暗硅有利于各种专用硬件单元，这些硬件单元的运行效率比通用处理器高得多；对特定计算无用的单元被禁用，以支持其他更有用的单元。随着我们进入暗硅时代，DBMS的持续成功将取决于它利用专用硬件的能力。然而，与许多高性能计算领域不同，数据库引擎对硬件加速有很大的抵抗力。数据库引擎必须再次自我改造，这一次要变得更加硬件友好。该项目旨在从头开始重新检查数据库引擎，围绕定制的硬件支持重新设计最昂贵和耗时的数据库操作；类似的策略已经在操作系统和虚拟机等领域取得了重大进展。由此产生的硬件-软件共生将赋予DBMS前所未有的效率和性能，它需要在功率受限的世界中收集和处理数据。

Database technology forms a critical underpinning of modern infrastructure, supporting key fields such as banking, commerce, internet services, governments and travel. Database management systems (DBMS) are successful because they provide a stable interface for applications to target, while continually reinventing themselves to exploit new hardware capabilities and tame a growing deluge of data. Unfortunately, a looming hardware crisis, known as "dark silicon," threatens our ability to control the digital fire hose. Power constraints and limitations in achievable parallelism are forcing systems to idle an increasing fraction of their compute resources at any given time; increasingly, application performance will plateau unless we can reduce the energy that computations consume. Such reductions come from speeding up a computation (so tasks finish sooner) or by improving power efficiency (so more tasks can run at once). Clever software techniques can help, but general purpose processors - by their very generality - impose severe limits on the gains software can achieve. Dark silicon thus favors a variety of specialized hardware units capable of operating at vastly higher efficiencies than general purpose processors; units not useful to a particular computation are disabled in favor of other, more useful ones. As we enter the dark silicon era, the continued success of the DBMS will thus hinge on its ability to utilize specialized hardware. Unlike many areas of high-performance computing, however, database engines are quite resistant to hardware acceleration. The database engine must again reinvent itself, this time to become more hardware-friendly. This project aims to re-examine database engines from the ground up, redesigning the most expensive and time consuming database operations around tailor-made hardware support; similar strategies have already yielded significant advances in areas such as operating systems and virtual machines. The resulting hardware-software symbiosis will endow the DBMS with the unprecedented efficiency and performance it needs to collect and process data in a power-constrained world.