Collaborative Research: Forecast Evaluation and Model Selection in the Presence of Structural Instability

宏观经济和金融数据不稳定性的普遍实证结果最近引起了广泛关注。因此，许多人致力于设计新的和改进的参数不稳定性测试，研究人员在他们的实证工作中更加关注这些工具。然而，在这种不稳定的环境中，允许预测模型评估和选择的方法仍然缺乏文献。因此，必须开发此类工具，调查人员的研究议程旨在填补这一空白。研究人员建议研究新的方法来评估经济模型的预测性能，并在存在结构不稳定的情况下进行模型选择。他们的方法的新奇是允许不稳定的环境中的预测模型的性能，以及竞争模型的相对性能，可能会随着时间的推移而变化。在第一个子项目，检测和预测故障，调查人员建议调查的理论框架，以评估是否预测模型估计在一个时期可以提供良好的预测在随后的时期。他们将预测故障定义为模型的样本外性能（由某些损失函数判断）明显劣于其样本内性能的情况，并提出了一种检测此类情况的方法。在第二个研究项目中，非嵌套模型选择不稳定的环境中，研究人员计划考虑非嵌套模型选择测试中可能存在的数据和参数不稳定性。他们的方法的新奇在于，它允许模型的相对性能随时间变化，而现有的模型选择技术则会寻找一个整体最佳模型。 拟议活动的更广泛影响将来自与科学界和学生的从业者分享的新方法，以及对我们理解经济学中不稳定性作用的贡献。 该项目为研究生提供指导，合作机会，论文动机和经济支持。该提案将产生将在研讨会和专业会议上提交的论文，并最终在学术期刊上发表。

The widespread empirical findings of instabilities in macroeconomic and financial data have attracted a lot of attention recently. As a result, much effort has been devoted to designing new and improved tests for parameter instability, and researchers have paid more attention to such tools in their empirical work. However, methods that allow forecast model evaluation and selection in such unstable environments are still lacking in the literature. It is therefore important to develop such tools, and the investigators' research agenda aims at filling that void. The investigators' propose to investigate new methods for evaluating the forecasting performance of economic models, and for conducting model selection in the presence of structural instability. The novelty of their approach is to allow for unstable environments where the forecasting performance of a model, as well as the relative performance of competing models, could be changing over time.In the first subproject, Detecting and Predicting Forecast Breakdowns, the investigators propose to investigate a theoretical framework for assessing whether a forecast model estimated over one period can provide good forecasts over a subsequent period. They define a forecast breakdown as a situation in which the out-of-sample performance of the model, judged by some loss function, is significantly worse than its in-sample performance, and propose a method to detect such situations. In a second research project, Non-nested Model Selection in Unstable Environments, the investigators plan to consider non-nested model selection tests in the presence of possible data and parameter instabilities. The novelty of their approach is that it allows the models' relative performance to be varying over time, whereas existing model selection techniques look for an overall best model. The broader impact of the proposed activity will come from the new methods shared with practitioners within the scientific community and students alike, and the contributions to our understanding the role of instabilities in economics. This project provides mentoring, collaboration opportunity, dissertation motivation and financial support for a graduate student. The proposal will result in papers that will be presented at seminars and professional conferences, and ultimately published in scholarly journals.