Quantitative model comparison for Time Series predictive models

随着越来越多的机器学习和人工智能技术被应用到决策过程中，成功控制和验证正成为现代数据科学最终用户越来越关注的问题。尤其令人担忧的是，对于新机型的表现是否比最先进的机型更好这样的问题，有可靠的答案。基于人工智能的决策过程真的比标准过程更好吗？人们寻求可复制的、定量的和可推广的自动化测试工作流。与此密切相关的是数据集成、流水线和建模的自动化构建元策略的问题，包括不确定性的预测和量化。该项目将在以下方面解决当前围绕模型验证和决策成功控制的主要挑战：i.定量模型比较和自动化模型构建的理论，重点是异类、结构化或分层建模任务，例如面板或时空数据。ii.自动化建模和模型验证工作流的软件工程，作为R和/或python的模块化工具箱环境实现。iii.在能源、工程、地理和医疗领域的实际应用中的部署，以及最佳实践和验证原则的发展。

With more and more machine learning and artificial intelligence technologies being adopted into decision making processes, success control and validation is becoming a matter of increasing concern for end users of modern data science. Of particular concern are reliable answers to questions such as 'does the new model perform better than the state-of-art one?' and 'is the artificial intelligence-based decision making process really better than the standard one', for which replicable, quantitative and generalizable automated testing workflows are sought. Closely related is the question of automated building meta-strategies for data integration, pipelining, and modelling including prediction making or quantification of uncertainty.This project will address the major contemporary challenges around model validation and successcontrol in decision making, in:I. Theory of quantitative model comparison and automated model building, with a focus onheterogeneous, structured or hierarchical modelling tasks such as for panel or spatio-temporal data.II. Software engineering of automated modelling and model validation workflows implementedas modular toolbox environments in R and/or python.III. Deployment in real world applications in the energy, engineering, geography and healthdomains, and development of best practices and validation principles.