NOVEL APPROACHES TO COMPARING THE PREDICTIVE ACCURACY OF NESTED MODELS IN DATA RICH AND HETEROGENEOUS PREDICTOR ENVIRONMENTS

比较竞争统计模型的样本外预测准确性是数据科学的重要组成部分，也是选择合适的规范以产生预测或区分竞争假设的关键指标。与通常通过样本内拟合优度测量和规格测试来评估的这些模型的解释能力不同，预测准确性和预测建模关注的是模型如何很好地处理看不见的数据并对一些感兴趣的结果产生准确的预测。该项目的目的是开发一个新的工具包，用于比较由两个或多个嵌套预测回归模型产生的时间序列预测的相对准确性，最终目标是检测可预测性或缺乏可预测性的关键驱动因素。我们考虑一个环境，在这个环境中，人们不仅要面对潜在的大量预测因子，而且这些预测因子允许显示动态特征的混合物，其中一些（或全部）是高度持久的，而另一些则是嘈杂的，因为它通常发生在经济和金融数据中。例如，一位对GDP增长预测感兴趣的宏观经济学家面临着数百个可能有用的预测指标，从记忆很少的嘈杂指标（如金融回报）到记忆更长的持久序列（如利率等趋势行为）。在预测准确性竞赛中将这些预测器捆绑在一起，或者完全忽略数据的持久性属性，都可能影响推断的可靠性，而不管这样的预测器是少还是多。尽管这些场景在应用工作中的相关性和无所不在，但预测准确性测试文献很少关注这些考虑因素。这项研究的新颖方面涉及实施预测准确性比较的具体标准，这将大大简化和推广现有的方法，以及它们可以应用的更丰富的环境。此外，在实证研究或政策分析过程中，研究人员往往面临着比较简单模型与较复杂模型的预测能力的问题，而简单模型是较复杂模型的特例。这样的模型对通常被称为嵌套的，而没有这种相似性的模型对被称为非嵌套的。嵌套模型是实证研究中最常见的设置之一，有助于回答基本问题，如：包含一组额外的预测因子是否显着提高了较小模型或不可预测基准的预测能力？无论一个人是在具有异质预测器类型的大数据环境中工作，还是在具有少数表现良好和纯粹平稳预测器的更理想的环境中工作，在嵌套模型之间进行样本外预测精度比较会提出许多技术挑战，尽管有大量关于该主题的文献，但这些挑战也没有以令人满意的方式得到解决（例如，在相同预测精度的假设下，两个嵌套模型崩溃为相同的规范，这一事实通常会导致具有退化方差的定义不清的测试统计）。这个建议的首要目标是引入一个全新的技术框架，可以适应模型之间的预测精度比较，而不管它们是否具有嵌套结构。然后，该框架将用于开发一个工具包，用于在数据丰富的环境中进行预测准确性测试和预测器筛选。

Comparing the out of sample predictive accuracy of competing statistical models is an essential component of data science and a key metric for choosing a suitable specification for the purpose of generating forecasts or discriminating between competing hypotheses. Unlike the explanatory power of such models which is commonly evaluated via in-sample goodness of fit measures and specification tests, predictive accuracy and predictive modelling are instead concerned with how well models can cope with unseen data and produce accurate forecasts of some outcome of interest. The purpose of this project is to develop a novel toolkit for comparing the relative accuracy of time series forecasts produced by two or more nested predictive regression models with the end-goal of detecting key drivers of predictability or the lack of it. We consider an environment where one is confronted with not only a potentially large pool of predictors but also with these predictors allowed to display a mixture of dynamic characteristics, some (or all) being highly persistent and others noisier as it commonly occurs in economic and financial data. A macroeconomist interested in forecasts of GDP growth for instance faces hundreds of potentially useful predictors ranging from noisy indicators with very little memory such as financial returns to more persistent series with much longer memory or trending behaviours such as interest rates. Bundling such predictors together in a predictive accuracy contest or ignoring the persistence properties of the data all-together is likely to affect the reliability of inferences regardless of whether there are few or many such predictors. Despite the relevance and omnipresence of such scenarios in applied work the predictive accuracy testing literature has devoted little attention to such considerations. The novel aspects of this research concern both the specific criteria introduced for implementing predictive accuracy comparisons which will considerably simplify and generalise existing approaches and the richer environment under which they can be applied. Furthermore and in the course of empirical research or policy analysis, researchers are often faced with the need to compare the forecasting ability of a simple model with a more complicated one, with the simple model being a special case of the more complicated model. Such model pairs are typically referred to as nested while model pairs with no such similarities are referred to as non-nested. Nested models are one of the most commonly encountered setting in empirical research and help answer fundamental questions such as: does the inclusion of a set of additional predictors significantly improve the predictive power of a smaller model or a non-predictability benchmark? Irrespective of whether one operates in a big data environment combined with heterogeneous predictor types or in a more idealised environment with few well behaved and purely stationary predictors, conducting out of sample predictive accuracy comparisons between nested models raises many technical challenges that have also not been resolved in a satisfactory way despite a voluminous literature on the subject (e.g. the fact that two nested models collapse into the same specification under the hypothesis of equal predictive accuracy typically results in ill-defined test statistics with degenerate variances). The overarching objective of this proposal is to introduce a totally new technical framework that can accommodate predictive accuracy comparisons between models irrespective of whether they have a nested structure or not. This framework will then be used to develop a toolkit for conducting predictive accuracy tests and predictor screening in data rich environments.