Collaborative Research: Honest and Robust Inference with High Dimensional Data

经济学和其他科学中的现代数据分析经常涉及这样的情况：研究人员有兴趣预测或解释许多个体的行为；研究人员也可能对单一因素如何影响结果感兴趣，但必须包括几个其他因素来解释感兴趣的结果。在这些情况下，对测量效果的统计不确定性进行测试是困难的，现有的方法不能提供精确和有效的方法来测量这种不确定性。该研究项目将开发新的和改进的方法，以获取与两种情况下的效果测量相关的统计不确定性。这些情况在经济学和其他社会科学研究中经常出现，这些研究涉及许多政策问题，而评估不确定性的准确方法对于提供良好的政策建议非常重要。这一研究项目的结果将有助于提高政策评价和统计预测的质量，从而为决策者提供更好的建议。这将有助于改善政策结果，从而促进美国经济更快增长。该研究项目将开发新的方法来测试两种情况下高维数据估计中的统计不确定性。在第一种设置中，我们开发了一种构造满足平均覆盖属性的区间的一般方法：区间覆盖平均真实效果的预先指定的分数（例如95%）。关注平均覆盖率使我们能够形成自动反映数据驱动的正则化效率增益的区间，包括经验贝叶斯方法，以及基于机器学习技术的回归函数估计器。在通常的覆盖概念下，这样的收益是不可能的，在这种概念下，需要对每一种影响单独进行覆盖保证，而不仅仅是平均覆盖。在第二种设置中，我们关注通常的覆盖概念。为了获得信息置信区间，我们利用对控制系数大小的先验限制。我们证明了我们的构造具有几个最优性和近最优性性质。这项研究的结果将提高政策建议的质量，从而提高美国的经济增长率。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Modern data analysis in economics and other sciences often involves situations in which researchers are interested in making predictions about or explaining the behavior of many individuals; researchers may also be interested in how a single factor affects an outcome but have to include several other factors in explaining the outcome of interest. Testing for statistical uncertainty surrounding the measured effects in these situations is difficult and existing methods do not provide precise and efficient ways to measure this uncertainty. This research project will develop new and improved methods for accessing statistical uncertainty associated with effect measurements in both situations. These situations arise routinely in economics and other social science research that addressing many policy questions, and accurate methods for assessing uncertainty are important for providing good policy recommendations. The results of this research project will help improve the quality of policy evaluation and statistical prediction and as result, provide policy makers with better advice. This will contribute to better policy outcomes, hence foster faster economic growth in the US.This research project will develop new methods to test statistical uncertainty in high-dimension data estimation in two settings. In the first setting, we develop a general method for constructing intervals that satisfy an average coverage property: the intervals cover a prespecified fraction (95%, say) of the true effects on average. Focusing on average coverage allows us to form intervals that automatically reflect efficiency gains from data-driven regularization, including empirical Bayes methods, and estimators of a regression function based on machine learning techniques. Such gains are not possible under the usual notion of coverage, under which a coverage guarantee is required for each effect individually, not just on average. In the second setting, we focus on the usual notion of coverage. To obtain informative confidence intervals, we exploit a priori restrictions on the magnitude of the control coefficients. We show that our construction enjoys several optimality and near-optimality properties. The results of this research will improve the quality of policy advice and as a result increase the rate of economic growth in the US.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.