Self-adjusting Model-based Processing of Declarative Forecast Queries in Data-Warehouse-Systems

时间序列数据的预测在许多领域的决策过程中都是至关重要的。在DFG项目FFQ(Flash-Foreward Query框架)中，我们开发了在关系数据库管理系统中集成自动时间序列预测的概念。与允许查看过去数据的闪回查询不同，我们提供声明性闪回查询，允许查看未来的数据。复杂的内部操作，如自动选择、使用和维护，对数据库用户是隐藏的。预测查询的自动透明处理基于基于相似性的模型选择方法、逻辑集成模型和配置顾问，配置顾问为多维数据集选择预测模型的配置并将其存储在模型池中，由于典型数据仓库环境中搜索空间大、参数估计时间长以及可能存在大数据集，因此维护此类模型配置是一项重大挑战。在每次新插入之后同步维护所有模型和配置是不可行的，如果模型精度仍然足够或者如果模型没有被查询引用，则甚至可能不需要。异步维护技术已经应用于实例化视图，通常会延迟维护以释放周期或直到查询引用某个视图。然而，与物化视图相比，预测模型总是产生估计的结果，因此在模型维护方面提供了更多的自由度，但也带来了更多的挑战。一方面，如果精确度仍然足够(取决于预测查询的要求)，则可以延迟甚至省略维护。另一方面，投机性维护可能会提供更好的解决方案，并导致更高的准确性。在这个后续项目中，我们的目标是开发一种灵活的调度机制来维护预测模型配置，该机制可以在不影响查询运行时间的情况下，对任意预测查询提供高精度。必须根据数据特征、模型准确性、查询工作负载和可用资源来计划维护任务。然而，只有在优化使用底层硬件环境并利用并行化机会的情况下，才有可能实现有效的维护。这需要维护任务的联合执行和并行化技术，以及能够生成维护任务和分配资源的新型成本模型。

Forecasting of time series data is crucial for decision-making processes in many domains. Within the DFG project FFQ (Flash-Foreward Query Framework) we developed a concept for integrating automatic time series forecasting in relational database management systems. In contrast to flash-back queries, which allow a view on the data in the past, we provide declarative flash-forward queries, which allow a view on the data in the future. Complex internals like automatic model selection, usage and maintenance are hidden from the database user. The automatic and transparent processing of such forecast queries is based on a similarity-based model selection approach, logical ensemble models and a configuration advisor, which selects a configuration of forecast models for multi-dimensional data sets and stores it in a model pool.Due to the large search space, long parameter estimation runtime and possible large data sets in typical data warehouse environments, the maintenance of such model configurations exhibits a major challenge. Synchronous maintenance of all models and configurations after each new insert is infeasible and might not even be necessary if the model accuracy is still sufficient or if a model is not referenced by a query. Asynchronous maintenance techniques were already applied for materialized views and usually delay maintenance to free cycles or until a view is referenced by a query. However, in contrast to materialized views, forecast models always produce estimated results and, thus, provide more freedom but also more challenges in model maintenance. On the one hand, maintenance can be delayed or even omitted if the accuracy is still sufficient (depending on the requirements of a forecast query). On the other hand, speculative maintenance might provide even better solutions and lead to higher accuracy. In this follow-up project we aim to develop a flexible scheduling mechanism for maintaining forecast model configurations, which allows high accuracy for arbitrary forecast queries without affecting query runtime. Maintenance tasks have to be scheduled based on data characteristics, model accuracy, the query workload, and available resources. However, efficient maintenance is only possible if the underlying hardware environment is optimal used and parallelization opportunities are exploited. This requires techniques for joint execution and parallelization of maintenance tasks and novel cost models that enable the generation of maintenance jobs and the distribution of resources.