CAREER: Distribution-Free and Adaptive Statistical Inference

近年来，科学学科之间拥抱复杂建模和黑盒机器学习算法的趋势越来越大。尽管在处理复杂的数据结构和拟合复杂的回归函数方面取得了显著的成功，但在将严格的统计原理整合到这些管道中方面仍然存在很大的差距。主要困难在于在不人为简化这些先进工具所固有的复杂性的情况下，实现可靠的不确定性量化和稳健的统计推断。大多数旨在弥补这一差距的现有框架都依赖于强假设，在这些假设下，机器学习算法可以准确地估计数据生成分布。然而，这些假设往往很难证明是合理的，特别是对于尚未完全理解的现代机器学习算法。这项研究项目旨在开发新的统计推理框架，涵盖任何机器学习算法或复杂模型，而不考虑故障模式。由此产生的方法能够解决黑盒机器学习算法在包括医学、医疗保健、经济学、政治学、流行病学和气候科学在内的广泛应用领域中对推理有效性的潜在威胁。还将开发开放源码软件，以帮助应用研究人员在不影响现代工具在非推理任务中的有效性的情况下，将严格的统计推断整合到他们特定领域的建模工作流程中。这可能会进一步缓解采用现代机器学习方法的犹豫，并促进科学和工程领域之间的合作。在整个项目中，PI将指导本科生和研究生，使他们对统计原理有扎实的理解，以便在面对快速发展的机器学习技术时成为未来的领导者。这项建议将侧重于无分布推理，它不受参数模型的错误指定、违反非参数假设(如光滑性或形状约束)、由于样本量有限、高维、边界情况或不规则性而导致的渐近近似的不准确。为了避免做出没有信息的决策，理想的无分布推理框架也应该适用于良好的建模。这意味着它应该和其他依赖于分布假设的框架一样高效。适应性缓解了健壮性和效率之间的权衡。PI将为三个具体问题开发无分发和自适应的推理框架。首先，在因果推理中，通过合并处理前的协变量，可以对部分识别的因果效应获得更紧密的识别集。然而，现有的尖锐推理框架需要估计给定协变量的潜在结果的条件分布。PI将开发一个基于对偶理论的通用框架，该框架能够围绕条件分布的任何估计，并进行无分布和自适应推理。其次，医学、政治经济学和因果推理中的许多目标参数可以通过给定协变量的结果的条件预期的极值来表示。与强加分布假设以实现对条件期望的一致估计的经典方法不同，PI将开发一个无分布的框架来测试统计零假设并直接在极值上构建有效的可信区间。最后，复杂模型和预测算法在时间序列预测和预测中的使用给可靠的不确定性量化带来了挑战。为了解决这一问题，PI将开发一个基于模型预测控制和保形预测的框架，该框架能够围绕任何预测算法并对其进行校准，以实现长期覆盖，而不需要对时间序列的分布做出任何假设。这项研究的最终目标是带来洞察力并提供一套工具，以增强机器学习的统计推理能力，并使用统计推理增强机器学习。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Recent years have witnessed a growing trend across scientific disciplines to embrace complex modeling and black-box machine learning algorithms. Despite the remarkable success of handling complex data structures and fitting sophisticated regression functions, there remains a substantial gap regarding the integration of rigorous statistical principles into these pipelines. The main difficulty revolves around achieving reliable uncertainty quantification and robust statistical inference without artificially simplifying the complexity inherent in these advanced tools. Most existing frameworks that aim to bridge the gap rely on strong assumptions under which the machine learning algorithm can accurately estimate the data generating distribution. Nevertheless, these assumptions are often hard to justify, especially for modern machine learning algorithms that have yet to be fully understood. This research project aims to develop new frameworks for statistical inference that wrap around any machine learning algorithms or complex models without concerning about failure modes. The resulting methods are able to address the potential threats to inferential validity caused by black-box machine learning algorithms in a wide range of applied fields, including medicine, healthcare, economics, political science, epidemiology, and climate sciences. Open source software will also be developed to help applied researchers integrate rigorous statistical inference into their domain-specific modeling workflows without compromising the effectiveness of modern tools in non-inferential tasks. This may further alleviate hesitation in adopting modern machine learning methods and catalyze collaboration between scientific and engineering fields. Throughout the project, the PI will mentor undergraduate and graduate students, equipping them with solid understandings of statistical principles to become future leaders in face of rapidly evolving machine learning techniques.This proposal will focus on distribution-free inference, which is immune to misspecification of parametric models, violation of nonparametric assumptions like smoothness or shape constraints, inaccuracy of asymptotic approximations due to limited sample size, high dimensionality, boundary cases, or irregularity. To avoid making uninformative decisions, an ideal distribution-free inference framework should also be adaptive to good modeling. This means that it should be as efficient as other frameworks that rely on distributional assumptions. Adaptivity alleviates the tradeoff between robustness and efficiency. The PI will develop distribution-free and adaptive inference frameworks for three specific problems. First, in causal inference, tighter identified set can be obtained for partially identified causal effects by incorporating pre-treatment covariates. However, existing frameworks for sharp inference require estimating conditional distributions of potential outcomes given covariates. The PI will develop a generic framework based on duality theory that is able to wrap around any estimates of conditional distributions and make distribution-free and adaptive inference. Second, many target parameters in medicine, political economy, and causal inference can be formulated through extremums of the conditional expectation of an outcome given covariates. In contrast to classical methods that impose distributional assumptions to enable consistent estimation of the conditional expectation, the PI will develop a distribution-free framework for testing statistical null hypotheses and constructing valid confidence intervals on the extremums directly. Finally, the use of complex models and prediction algorithms in time series nowcasting and forecasting presents challenges for reliable uncertainty quantification. To address this, the PI will develop a framework based on model predictive control and conformal prediction that is able to wrap around any forecasting algorithms and calibrate it to achieve long-term coverage, without any assumptions on the distribution of the time series. The ultimate goal of this research is to bring insights and present a suite of tools to empower statistical reasoning with machine learning and augment machine learning with statistical reasoning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.