Model Selection Diagnostics and Localized Model Selection/Combination

虽然过去十年的研究使人们普遍认识到由于模型选择而导致的统计不确定性的严重性，但需要付出更多的努力来改革目前仍然占主导地位的将所有统计结论建立在最终选择的模型上的做法。从方法论的角度来看，模型选择和模型组合工具箱中缺少的一个关键组件是模型选择诊断（而不是模型诊断）。PI寻求超越简单的自举不确定性度量的模型选择诊断方法。他们将通过考虑变量子集之间的距离或候选模型之间的估计的方法来解决变量选择和感兴趣数量估计中的不确定性。对于高维或复杂的数据，很可能不同的候选过程在不同的区域表现最好，特别是当考虑到非常不同的学习方法时。这需要本地化的模型选择/组合方法和理论，这是这个项目的第二个主要组成部分。PI采用了新的方法，并对局部模型选择或组合的新方法的性能得出了oracle不等式。统计方法已成为所有应用科学的基本组成部分。在自然和社会现象的数学描述中，对真正的不确定性进行适当的量化，对于得出公正和准确的结论至关重要。由于模型选择和模型组合在统计分析中起着核心作用，因此提出的准确测量模型选择不确定性的工作以及由此产生的更好的模型选择和模型组合工具，以及该领域的其他研究，有望为改变目前应用科学中统计模型选择的不合理做法做出重大贡献。在信息提取中更好地利用数据将对科学研究、政策和决策产生更广泛的影响。

Although research in the last decade has brought in general awareness of the seriousness of statistical uncertainty due to model selection, much more effort is needed to reform the currently still dominating practice of basing all statistical conclusions on a final selected model. From a methodological standpoint, a critical component missing in the toolbox of model selection and model combination is model selection diagnostics (not model diagnostics). The PI seeks model selection diagnosis methods that go beyond simple bootstrap uncertainty measures. They will address the uncertainty in variable selection and in estimation of a quantity of interest via means that take into account the distances between subsets of variables or between estimates from the candidate models. For high-dimensional or complex data, it is very likely that different candidate procedures perform the best in different regions, especially when very distinct learning methods are considered. This calls for localized model selection/combination methodology and theory, which is the second major component of this project. The PI takes new approaches and derives oracle inequalities on performance of the new methods for localized model selection or combination.Statistical methods have become an essential ingredient in all applied sciences. Proper quantifications of the true uncertainty in mathematical descriptions of the natural and social phenomena are fundamentally important to draw unbiased and accurate conclusions. Since model selection and model combination play a central role in statistical analysis, the proposed work on accurately measuring model selection uncertainty and the resulting better tools for model selection and model combination, together with other researches in the area, are expected to contribute substantially to changing the currently unsound practices of statistical model selection in applied sciences. The improved use of data in information extraction will have broader impacts in scientific research, policy and decision making.