Data Partitioning via Deep Reinforcement Learning

在分区数据库环境中，数据库中的表可以位于一个或多个数据库分区中。除了使数据存储更具可伸缩性外，分区数据库还允许在多个分区上并行执行查询，从而提供显著的性能优势。但是，不同的数据库分区方式可能会对查询处理性能产生截然不同的影响。例如，对查询工作负载中频繁联接的两个大表进行共同分区可能有助于极大地降低通过网络传输数据所产生的通信成本。然而，找到数据库的最佳分区并不是一件容易的事。如今，大多数数据库管理系统(DBMS)都依赖用户来指定对哪些表进行分区以及如何对表进行分区。尽管数据库社区最近取得了旨在自动化物理数据库设计的进展，但数据库分区的可行解决方案仍然难以找到。通过这个项目，我们提出利用深度强化学习来帮助解决这个问题。我们的目标是在大规模并行处理(MPP)环境中提供更快、更好的分区决策，以取代当前的DBMS分区顾问，从而使数据库管理员不必手动做出这样的决策。

In a partitioned database environment, tables in a database can be located in one or more database partitions. Aside from making data storage more scalable, a partitioned database makes it possible to execute queries over multiple partitions in parallel, thus providing significant performance advantages. However, different ways of partitioning the database may have drastically different implications on query processing performance. For example, co-partitioning two large tables that are frequently joined in the query workload may help vastly reduce the communication cost incurred by data transfer over the network. However, finding the optimal partitioning of a database is non-trivial. Most database management systems (DBMS) today rely on users to specify which and how tables are partitioned. Despite recent advances in the database community that aim to automate the physical database design, a viable solution to the database partitioning is still elusive. Through this project, we propose to utilize deep reinforcement learning to help solve this problem. We aim to provide faster and better partitioning decisions in replace of the current partition advisor for DBMS in a Massively Parallel Processing (MPP) setting, relieving the database administrators from making such decisions manually.