权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

NOVEL APPROACHES TO COMPARING THE PREDICTIVE ACCURACY OF NESTED MODELS IN DATA RICH AND HETEROGENEOUS PREDICTOR ENVIRONMENTS

比较数据丰富且异构预测器环境中嵌套模型的预测准确性的新方法

基本信息

批准号：
ES/W000989/1
负责人：
Jean-Yves Pitarakis
金额：
$ 40.19万
依托单位：
University of Southampton
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2021
资助国家：
英国
起止时间：
2021 至无数据
项目状态：
未结题

来源：
https://gtr.ukri.org/projects?ref=ES%2FW000989%2F1
关键词：
NOVEL APPROACHES COMPARING PREDICTIVE ACCURACY

项目摘要

Comparing the out of sample predictive accuracy of competing statistical models is an essential component of data science and a key metric for choosing a suitable specification for the purpose of generating forecasts or discriminating between competing hypotheses. Unlike the explanatory power of such models which is commonly evaluated via in-sample goodness of fit measures and specification tests, predictive accuracy and predictive modelling are instead concerned with how well models can cope with unseen data and produce accurate forecasts of some outcome of interest. The purpose of this project is to develop a novel toolkit for comparing the relative accuracy of time series forecasts produced by two or more nested predictive regression models with the end-goal of detecting key drivers of predictability or the lack of it. We consider an environment where one is confronted with not only a potentially large pool of predictors but also with these predictors allowed to display a mixture of dynamic characteristics, some (or all) being highly persistent and others noisier as it commonly occurs in economic and financial data. A macroeconomist interested in forecasts of GDP growth for instance faces hundreds of potentially useful predictors ranging from noisy indicators with very little memory such as financial returns to more persistent series with much longer memory or trending behaviours such as interest rates. Bundling such predictors together in a predictive accuracy contest or ignoring the persistence properties of the data all-together is likely to affect the reliability of inferences regardless of whether there are few or many such predictors. Despite the relevance and omnipresence of such scenarios in applied work the predictive accuracy testing literature has devoted little attention to such considerations. The novel aspects of this research concern both the specific criteria introduced for implementing predictive accuracy comparisons which will considerably simplify and generalise existing approaches and the richer environment under which they can be applied. Furthermore and in the course of empirical research or policy analysis, researchers are often faced with the need to compare the forecasting ability of a simple model with a more complicated one, with the simple model being a special case of the more complicated model. Such model pairs are typically referred to as nested while model pairs with no such similarities are referred to as non-nested. Nested models are one of the most commonly encountered setting in empirical research and help answer fundamental questions such as: does the inclusion of a set of additional predictors significantly improve the predictive power of a smaller model or a non-predictability benchmark? Irrespective of whether one operates in a big data environment combined with heterogeneous predictor types or in a more idealised environment with few well behaved and purely stationary predictors, conducting out of sample predictive accuracy comparisons between nested models raises many technical challenges that have also not been resolved in a satisfactory way despite a voluminous literature on the subject (e.g. the fact that two nested models collapse into the same specification under the hypothesis of equal predictive accuracy typically results in ill-defined test statistics with degenerate variances). The overarching objective of this proposal is to introduce a totally new technical framework that can accommodate predictive accuracy comparisons between models irrespective of whether they have a nested structure or not. This framework will then be used to develop a toolkit for conducting predictive accuracy tests and predictor screening in data rich environments.

比较竞争统计模型的样本外预测准确性是数据科学的重要组成部分，也是选择合适规范以生成预测或区分竞争假设的关键指标。与通常通过样本内拟合优度测量和规格测试评估的此类模型的解释能力不同，预测准确性和预测建模关注的是模型如何科普看不见的数据并对某些感兴趣的结果进行准确预测。这个项目的目的是开发一个新的工具包，用于比较两个或多个嵌套预测回归模型产生的时间序列预测的相对准确性，最终目标是检测可预测性的关键驱动因素或缺乏它。我们考虑一个环境，其中一个不仅面临着一个潜在的大池的预测因子，而且这些预测因子允许显示混合的动态特性，一些（或全部）是高度持久的，而另一些则是噪声较大的，因为它通常出现在经济和金融数据中。例如，一个对GDP增长预测感兴趣的宏观经济学家面临着数百个潜在有用的预测指标，从记忆力很小的噪声指标（如财务回报）到记忆力更长的更持久的序列或趋势行为（如利率）。在预测准确性竞赛中将这些预测因子捆绑在一起或完全忽略数据的持久性属性可能会影响推断的可靠性，无论这些预测因子是少还是多。尽管在应用工作中这种情况的相关性和无处不在的预测准确性测试文献很少关注这样的考虑。这项研究的新方面关注的具体标准，实施预测精度比较，这将大大简化和概括现有的方法和更丰富的环境下，他们可以应用。此外，在实证研究或政策分析过程中，研究人员经常需要比较简单模型与较复杂模型的预测能力，而简单模型是较复杂模型的特例。这样的模型对通常被称为嵌套的，而没有这样的相似性的模型对被称为非嵌套的。嵌套模型是实证研究中最常见的设置之一，有助于回答一些基本问题，例如：包含一组额外的预测因子是否会显着提高较小模型或不可预测基准的预测能力？无论是在与异构预测器类型相结合的大数据环境中操作，还是在具有很少表现良好和纯静态预测器的更理想化的环境中操作，在嵌套模型之间进行样本外预测精度的比较提出了许多技术挑战，尽管关于该主题的大量文献，这些挑战也没有以令人满意的方式得到解决（例如，两个嵌套模型在相同预测精度的假设下崩溃为相同规格的事实通常会导致具有退化方差的定义不清的检验统计量）。该提案的总体目标是引入一个全新的技术框架，该框架可以容纳模型之间的预测准确性比较，无论它们是否具有嵌套结构。然后，该框架将用于开发一个工具包，用于在数据丰富的环境中进行预测准确性测试和预测筛选。