Data Partitioning via Deep Reinforcement Learning

通过深度强化学习进行数据分区

• 批准号: 571801-2021
• 负责人: Yu, XiaohuiX
• 金额: $ 1.47万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Alliance Grants
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=757244
• 关键词: Data; Partitioning; via; Deep; Reinforcement

In a partitioned database environment, tables in a database can be located in one or more database partitions. Aside from making data storage more scalable, a partitioned database makes it possible to execute queries over multiple partitions in parallel, thus providing significant performance advantages. However, different ways of partitioning the database may have drastically different implications on query processing performance. For example, co-partitioning two large tables that are frequently joined in the query workload may help vastly reduce the communication cost incurred by data transfer over the network. However, finding the optimal partitioning of a database is non-trivial. Most database management systems (DBMS) today rely on users to specify which and how tables are partitioned. Despite recent advances in the database community that aim to automate the physical database design, a viable solution to the database partitioning is still elusive. Through this project, we propose to utilize deep reinforcement learning to help solve this problem. We aim to provide faster and better partitioning decisions in replace of the current partition advisor for DBMS in a Massively Parallel Processing (MPP) setting, relieving the database administrators from making such decisions manually.

在分区数据库环境中，数据库中的表可以位于一个或多个数据库分区中。除了使数据存储更具可伸缩性之外，分区数据库还可以并行地在多个分区上执行查询，从而提供显著的性能优势。然而，不同的数据库分区方式可能对查询处理性能有着截然不同的影响。例如，对查询工作负载中频繁联接的两个大表进行共同分区，可以帮助极大地减少通过网络传输数据所产生的通信成本。然而，找到数据库的最佳分区是不平凡的。目前，大多数数据库管理系统（DBMS）都依赖于用户来指定哪些表以及如何分区。尽管数据库社区最近取得了旨在自动化物理数据库设计的进展，但数据库分区的可行解决方案仍然难以捉摸。通过这个项目，我们建议利用深度强化学习来帮助解决这个问题。我们的目标是提供更快，更好的分区决策，以取代目前的分区顾问DBMS在大规模并行处理（MPP）设置，减轻数据库管理员从手动作出这样的决定。