Machine learning based optimization for database workloads

基于机器学习的数据库工作负载优化

• 批准号: 543957-2019
• 负责人: Litoiu, Marin
• 金额: $ 4.95万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Collaborative Research and Development Grants
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=691437
• 关键词: Machine; learning; based; optimization; database

This project aims to investigate machine learning and self-optimization algorithms for database workloads. The focus is on techniques that can enable run-time adaptive reconfiguration through workload classification, performance prediction and resource planning and execution. Current state-of-the-art approaches adopted in the industry reside on a number of static memory models implemented for each sort of consuming query operator in the access plan (e.g. sort, hash join, etc.). Each model is provided with different inputs depending on the operator type (e.g. cardinality, number of columns, column widths, and various other statistics) and computes an estimate of the memory that would be used by the operator. While these models were built with an understanding of runtime behavior, they are static models, not sufficiently accurate and not enabled for self-tuning and self-optimization. As a result, when runtime implementation of an operator or the operating environment changes, then the models become outdated. In contrast to existing approaches, this project looks at a broad spectrum of deep learning models for prediction and at adaptive look-ahead optimization to account for accurate, cost-effective runtime decisions. The first goal of the project is to develop and test new machine learning models for database workload estimation. The second complementary goal is to develop new model based self-optimization algorithms for memory, computing and storage allocation. The research will generate knowledge and innovation that yield (i) better system utilization and less cost and (ii) better system performance and stability. At the same time, the project will prepare a new generation of researchers and developers, trained in machine learning, database systems and cloud computing.The partner in the project is IBM Canada Lab and IBM Centre for Advanced Studies.

该项目旨在研究数据库工作负载的机器学习和自优化算法。重点是技术，可以使运行时自适应重新配置，通过工作负载分类，性能预测和资源规划和执行。目前业界采用的最先进的方法依赖于为访问计划中的每种消费查询运算符（例如，排序、散列连接等）实现的多个静态存储器模型。每个模型根据运算符类型（例如基数、列数、列宽和各种其他统计数据）提供不同的输入，并计算运算符将使用的内存的估计值。虽然这些模型是在理解运行时行为的基础上构建的，但它们是静态模型，不够准确，也不能进行自调优和自优化。因此，当操作符的运行时实现或操作环境发生变化时，模型就会过时。与现有方法相比，该项目着眼于广泛的深度学习预测模型和自适应前瞻优化，以实现准确，经济高效的运行时决策。该项目的第一个目标是开发和测试用于数据库工作负载估计的新机器学习模型。第二个互补的目标是开发新的模型为基础的自优化算法的内存，计算和存储分配。该研究将产生知识和创新，产生（i）更好的系统利用率和更低的成本，（ii）更好的系统性能和稳定性。与此同时，该项目将培养新一代的研究人员和开发人员，在机器学习，数据库系统和云计算方面进行培训。该项目的合作伙伴是IBM加拿大实验室和IBM高级研究中心。