Collaborative Research: Forecast Evaluation and Model Selection in the Presence of Structural Instability

近年来，宏观经济和金融数据不稳定性的实证研究结果引起了人们的广泛关注。因此，人们投入了大量的精力来设计新的和改进的参数不稳定性测试，研究人员在他们的实证工作中更加关注这些工具。然而，在这种不稳定的环境中，允许预测模型评估和选择的方法在文献中仍然缺乏。因此，开发这样的工具是很重要的，研究人员的研究议程旨在填补这一空白。研究人员建议研究评估经济模型预测性能的新方法，并在结构不稳定的情况下进行模型选择。他们的方法的新颖之处在于允许不稳定的环境，其中一个模型的预测性能，以及竞争模型的相对性能，可能会随着时间的推移而变化。在第一个子项目“检测和预测预测故障”中，研究人员建议研究一个理论框架，用于评估在一个时期估计的预测模型是否可以在随后的时期提供良好的预测。他们将预测崩溃定义为由某些损失函数判断的模型的样本外性能明显低于样本内性能的情况，并提出了一种检测这种情况的方法。在第二个研究项目“不稳定环境中的非嵌套模型选择”中，研究人员计划在可能存在数据和参数不稳定的情况下考虑非嵌套模型选择测试。他们的方法的新颖之处在于，它允许模型的相对性能随时间变化，而现有的模型选择技术寻找一个整体上最好的模型。拟议活动的更广泛影响将来自与科学界从业者和学生分享的新方法，以及对我们理解不稳定性在经济学中的作用的贡献。该项目为研究生提供指导，合作机会，论文动机和经济支持。该提案将形成论文，将在研讨会和专业会议上发表，并最终发表在学术期刊上。

The widespread empirical findings of instabilities in macroeconomic and financial data have attracted a lot of attention recently. As a result, much effort has been devoted to designing new and improved tests for parameter instability, and researchers have paid more attention to such tools in their empirical work. However, methods that allow forecast model evaluation and selection in such unstable environments are still lacking in the literature. It is therefore important to develop such tools, and the investigators' research agenda aims at filling that void. The investigators' propose to investigate new methods for evaluating the forecasting performance of economic models, and for conducting model selection in the presence of structural instability. The novelty of their approach is to allow for unstable environments where the forecasting performance of a model, as well as the relative performance of competing models, could be changing over time.In the first subproject, Detecting and Predicting Forecast Breakdowns, the investigators propose to investigate a theoretical framework for assessing whether a forecast model estimated over one period can provide good forecasts over a subsequent period. They define a forecast breakdown as a situation in which the out-of-sample performance of the model, judged by some loss function, is significantly worse than its in-sample performance, and propose a method to detect such situations. In a second research project, Non-nested Model Selection in Unstable Environments, the investigators plan to consider non-nested model selection tests in the presence of possible data and parameter instabilities. The novelty of their approach is that it allows the models' relative performance to be varying over time, whereas existing model selection techniques look for an overall best model. The broader impact of the proposed activity will come from the new methods shared with practitioners within the scientific community and students alike, and the contributions to our understanding the role of instabilities in economics. This project provides mentoring, collaboration opportunity, dissertation motivation and financial support for a graduate student. The proposal will result in papers that will be presented at seminars and professional conferences, and ultimately published in scholarly journals.