Collaborative Research: Honest and Robust Inference with High Dimensional Data

经济学和其他科学中的现代数据分析通常涉及这样的情况：研究人员对预测或解释许多人的行为感兴趣；研究人员也可能对单个因素如何影响结果感兴趣，但在解释感兴趣的结果时，必须包括其他几个因素。在这些情况下，测试围绕测量的影响的统计不确定性是困难的，现有方法没有提供精确和有效的方法来衡量这种不确定性。这一研究项目将开发新的和改进的方法，以获取与这两种情况下的效果测量相关的统计不确定性。这些情况经常出现在经济学和其他社会科学研究中，这些研究涉及许多政策问题，而准确评估不确定性的方法对于提供良好的政策建议非常重要。本研究项目的成果将有助于提高政策评估和统计预测的质量，从而为政策制定者提供更好的建议。这将有助于更好的政策结果，从而促进美国更快的经济增长。本研究项目将开发新的方法，在两个背景下测试高维数据估计的统计不确定性。在第一个设置中，我们开发了一种构造满足平均覆盖属性的区间的一般方法：这些区间平均覆盖真实效果的预定分数(比如95%)。关注平均覆盖率允许我们形成区间，自动反映数据驱动的正则化的效率收益，包括经验贝叶斯方法和基于机器学习技术的回归函数估计器。在通常的覆盖范围概念下，这种收益是不可能的，在这种概念下，每种影响都需要单独的覆盖保证，而不仅仅是平均而言。在第二个设置中，我们将重点放在通常的覆盖概念上。为了得到信息可信区间，我们利用了对控制系数大小的先验限制。我们证明了我们的构造具有几个最优性和近最优性。这项研究的结果将提高政策建议的质量，从而提高美国的经济增长率。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Modern data analysis in economics and other sciences often involves situations in which researchers are interested in making predictions about or explaining the behavior of many individuals; researchers may also be interested in how a single factor affects an outcome but have to include several other factors in explaining the outcome of interest. Testing for statistical uncertainty surrounding the measured effects in these situations is difficult and existing methods do not provide precise and efficient ways to measure this uncertainty. This research project will develop new and improved methods for accessing statistical uncertainty associated with effect measurements in both situations. These situations arise routinely in economics and other social science research that addressing many policy questions, and accurate methods for assessing uncertainty are important for providing good policy recommendations. The results of this research project will help improve the quality of policy evaluation and statistical prediction and as result, provide policy makers with better advice. This will contribute to better policy outcomes, hence foster faster economic growth in the US.This research project will develop new methods to test statistical uncertainty in high-dimension data estimation in two settings. In the first setting, we develop a general method for constructing intervals that satisfy an average coverage property: the intervals cover a prespecified fraction (95%, say) of the true effects on average. Focusing on average coverage allows us to form intervals that automatically reflect efficiency gains from data-driven regularization, including empirical Bayes methods, and estimators of a regression function based on machine learning techniques. Such gains are not possible under the usual notion of coverage, under which a coverage guarantee is required for each effect individually, not just on average. In the second setting, we focus on the usual notion of coverage. To obtain informative confidence intervals, we exploit a priori restrictions on the magnitude of the control coefficients. We show that our construction enjoys several optimality and near-optimality properties. The results of this research will improve the quality of policy advice and as a result increase the rate of economic growth in the US.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.