Self-adjusting Model-based Processing of Declarative Forecast Queries in Data-Warehouse-Systems

数据仓库系统中基于自调整模型的声明性预测查询处理

• 批准号: 114523986
• 负责人: Professor Dr.-Ing. Wolfgang Lehner
• 金额: --
• 依托单位: Professur für Datenbanken
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2009
• 资助国家: 德国
• 起止时间: 2008-12-31 至 2018-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/114523986?language=en
• 关键词: Self; adjusting; Model; based; Processing

Forecasting of time series data is crucial for decision-making processes in many domains. Within the DFG project FFQ (Flash-Foreward Query Framework) we developed a concept for integrating automatic time series forecasting in relational database management systems. In contrast to flash-back queries, which allow a view on the data in the past, we provide declarative flash-forward queries, which allow a view on the data in the future. Complex internals like automatic model selection, usage and maintenance are hidden from the database user. The automatic and transparent processing of such forecast queries is based on a similarity-based model selection approach, logical ensemble models and a configuration advisor, which selects a configuration of forecast models for multi-dimensional data sets and stores it in a model pool.Due to the large search space, long parameter estimation runtime and possible large data sets in typical data warehouse environments, the maintenance of such model configurations exhibits a major challenge. Synchronous maintenance of all models and configurations after each new insert is infeasible and might not even be necessary if the model accuracy is still sufficient or if a model is not referenced by a query. Asynchronous maintenance techniques were already applied for materialized views and usually delay maintenance to free cycles or until a view is referenced by a query. However, in contrast to materialized views, forecast models always produce estimated results and, thus, provide more freedom but also more challenges in model maintenance. On the one hand, maintenance can be delayed or even omitted if the accuracy is still sufficient (depending on the requirements of a forecast query). On the other hand, speculative maintenance might provide even better solutions and lead to higher accuracy. In this follow-up project we aim to develop a flexible scheduling mechanism for maintaining forecast model configurations, which allows high accuracy for arbitrary forecast queries without affecting query runtime. Maintenance tasks have to be scheduled based on data characteristics, model accuracy, the query workload, and available resources. However, efficient maintenance is only possible if the underlying hardware environment is optimal used and parallelization opportunities are exploited. This requires techniques for joint execution and parallelization of maintenance tasks and novel cost models that enable the generation of maintenance jobs and the distribution of resources.

时间序列数据的预测对于许多领域的决策过程至关重要。在DFG项目FFQ（Flash Forward Query Framework）中，我们开发了一个概念，用于在关系数据库管理系统中集成自动时间序列预测。与允许查看过去数据的闪回查询相比，我们提供了声明式的闪回查询，允许查看未来的数据。像自动模型选择、使用和维护这样复杂的内部机制对数据库用户是隐藏的。这种预测查询的自动和透明处理是基于相似性的模型选择方法、逻辑集成模型和配置顾问，该配置顾问为多维数据集选择预测模型的配置并将其存储在模型池中。由于在典型的数据仓库环境中大的搜索空间、长的参数估计运行时间和可能的大数据集，这种模型配置的维护是一个重大挑战。在每次新插入之后同步维护所有模型和配置是不可行的，并且如果模型精度仍然足够或者如果模型没有被查询引用，则甚至可能不需要。异步维护技术已经应用于物化视图，并且通常延迟维护以释放周期或直到视图被查询引用。然而，与物化视图相比，预测模型总是产生估计的结果，因此，提供了更多的自由，但在模型维护方面也面临更多的挑战。 一方面，如果准确性仍然足够（取决于预测查询的要求），则可以延迟甚至省略维护。另一方面，投机性维护可能会提供更好的解决方案，并导致更高的准确性。在这个后续项目中，我们的目标是开发一个灵活的调度机制，用于维护预测模型配置，它允许任意预测查询的高准确性，而不影响查询运行时间。维护任务必须根据数据特征、模型准确性、查询工作负载和可用资源进行调度。 然而，只有在底层硬件环境得到最佳利用并利用并行化机会的情况下，才有可能进行有效的维护。这就需要技术的联合执行和并行化的维护任务和新的成本模型，使维护工作的生成和资源的分配。

项目成果

CSAR: the cross-sectional autoregression model for short and long-range forecasting

• DOI: 10.1007/s41060-018-00169-7
• 发表时间: 2019-01
• 期刊: International Journal of Data Science and Analytics
• 影响因子: 2.4
• 作者: Claudio Hartmann;F. Ressel;M. Hahmann;Dirk Habich;Wolfgang Lehner

CSAR: The Cross-Sectional Autoregression Model

CSAR：横截面自回归模型

• DOI: 10.1109/dsaa.2017.27
• 发表时间: 2017
• 期刊: 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA)
• 作者: Hartmann;Hahmann;Habich;Lehner
• 通讯作者: Lehner