Model Selection Diagnostics and Localized Model Selection/Combination

虽然过去十年的研究使人们普遍认识到由于模型选择而造成的统计不确定性的严重性，但仍需要作出更多的努力来改革目前仍然占主导地位的做法，即所有统计结论都以最终选定的模型为基础。从方法论的角度来看，模型选择和模型组合工具箱中缺少的一个关键组成部分是模型选择诊断（而不是模型诊断）。PI寻求超越简单自举不确定性测量的模型选择诊断方法。它们将通过考虑到变量子集之间或候选模型估计值之间的距离的方法来解决变量选择和感兴趣数量估计中的不确定性。对于高维或复杂的数据，很可能不同的候选过程在不同的区域中表现最好，特别是当考虑非常不同的学习方法时。这就需要本地化的模型选择/组合方法和理论，这是本项目的第二个主要组成部分。PI采用了新的方法，并推导出了关于局部化模型选择或组合的新方法的性能的预言不等式。统计方法已成为所有应用科学的重要组成部分。在自然和社会现象的数学描述中，正确量化真实的不确定性对于得出公正和准确的结论至关重要。由于模型选择和模型组合在统计分析中起着核心作用，准确测量模型选择的不确定性和由此产生的更好的工具，模型选择和模型组合，以及在该领域的其他研究，预计将大大有助于改变目前不健全的做法，统计模型选择在应用科学。在信息提取中更好地利用数据将对科学研究、政策和决策产生更广泛的影响。

Although research in the last decade has brought in general awareness of the seriousness of statistical uncertainty due to model selection, much more effort is needed to reform the currently still dominating practice of basing all statistical conclusions on a final selected model. From a methodological standpoint, a critical component missing in the toolbox of model selection and model combination is model selection diagnostics (not model diagnostics). The PI seeks model selection diagnosis methods that go beyond simple bootstrap uncertainty measures. They will address the uncertainty in variable selection and in estimation of a quantity of interest via means that take into account the distances between subsets of variables or between estimates from the candidate models. For high-dimensional or complex data, it is very likely that different candidate procedures perform the best in different regions, especially when very distinct learning methods are considered. This calls for localized model selection/combination methodology and theory, which is the second major component of this project. The PI takes new approaches and derives oracle inequalities on performance of the new methods for localized model selection or combination.Statistical methods have become an essential ingredient in all applied sciences. Proper quantifications of the true uncertainty in mathematical descriptions of the natural and social phenomena are fundamentally important to draw unbiased and accurate conclusions. Since model selection and model combination play a central role in statistical analysis, the proposed work on accurately measuring model selection uncertainty and the resulting better tools for model selection and model combination, together with other researches in the area, are expected to contribute substantially to changing the currently unsound practices of statistical model selection in applied sciences. The improved use of data in information extraction will have broader impacts in scientific research, policy and decision making.