CAREER: Integrating Optimal Design and Inference for Modern Observational Studies

本研究项目将开发新的方法来推断大型管理数据集中的因果关系。这些数据集日益成为卫生服务、公共政策和社会科学中因果关系的重要证据来源。测量感兴趣的治疗的因果效应的程序强调创建相似的个体亚组，一个接受治疗，另一个接受控制。然而，在实践中，这一过程并不能在大型管理数据集中实现完美的相似性，特别是当研究中的单元随时间或空间的变化而呈现结构时。待开发的方法将产生置信区间和假设检验，这些置信区间和假设检验明确地说明了单元的不完美设计和结构。对这些方法的仔细分析将为如何减少初始设计的不完美提供有价值的指导。由此产生的工具将在模块化框架中与现代机器学习方法有效配对，并将立即适用于健康和教育成果的大规模研究。支持的教育活动将吸引来自不同背景的本科生研究人员，培养研究生未来担任教师导师的角色，并制作对培养本科生因果推理原则有价值的教学材料。开源软件也将被开发。本项目将专注于整合两种广泛使用的因果推理设计的设计和推理，这些设计试图从最初不同的处理和控制样本中创建可信的比较：匹配，将相似的处理和控制个体组合在一起，形成小的同质匹配集；加权，它为学习单元构建权重，目的是淡化组间的差异，强调组间的相似之处。将承担四项具体任务。首先，现有的匹配设计的排列推理方法，即在匹配组中重新洗牌处理和控制单元的标签以构建假设检验，将通过允许排列概率根据单个匹配集中剩余差异的程度而变化来进行转换。由此产生的方法易于使用估计的治疗概率来实现，允许对未观察到的变量进行敏感性分析，并提出了一种选择初始匹配的新方法，该方法可以有效地管理治疗概率相似性和结果风险相似性之间的权衡。其次，将设计新的算法来有效地从条件排列分布中采样，这些分布尊重匹配中的设计约束，包括估计处理概率的最优配对和多变量的约束不平衡。这些工具将通过关注研究设计的各个方面来减少偏差并提高精度。第三，匹配排列推断的工具将扩展到集群观察研究，在集群和个体水平以及可能的溢出效应上进行处理。还将构建相应的敏感性分析，处理个体和集群水平上未观察到的变量。最后，我们将构建一种新的方法来衡量存在不可观测变量的加权方法的大样本性能，量化设计选择对不可观测变量的稳健性偏差的影响。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This research project will develop new methods for inference about causal relationships in large administrative datasets. These datasets are an increasingly important source of evidence about causal effects in health services, public policy, and the social sciences. Procedures for measuring the causal effect of a treatment of interest emphasize creating similar subgroups of individuals, one receiving treatment and another receiving control. In practice, however, this process is not able to achieve perfect similarity in large administrative datasets, especially when the units in the study exhibit structure over time or space. The methods to be developed will produce confidence intervals and hypothesis tests that account explicitly for imperfect design and structure over units. Careful analysis of these methods will lead to valuable guidance for how to make initial designs less imperfect. The resulting tools will pair effectively with modern machine learning methods in a modular framework and will immediately be applicable to large-scale studies of health and educational outcomes. Supported educational activities will engage undergraduate researchers from diverse backgrounds, groom graduate students for future roles as faculty mentors, and produce pedagogical materials valuable for training undergraduate students in the principles of causal inference. Open-source software also will be developed.This project will focus on integrating design and inference for two widely used causal inference designs that attempt to create credible comparisons from initially different treatment and control samples: matching, which groups similar treated and control individuals together into small homogenous matched sets; and weighting, which constructs weights for study units with the goal of downplaying dissimilarities and emphasizing similarities between the groups. Four specific tasks will be undertaken. First, existing methods of permutation inference for matched designs, which reshuffle labels for treated and control units within matched groups to construct hypothesis tests, will be transformed by allowing permutation probabilities to vary according to the degree of remaining discrepancy in individual matched sets. The resulting method, which will be easy to implement using estimated probabilities of treatment, permits sensitivity analysis for unobserved variables and suggests a new method of choosing an initial match that effectively manages tradeoffs between similarity on probability of treatment and similarity on outcome risk. Second, new algorithms will be designed to efficiently sample from conditional permutation distributions that respect design constraints in matching including optimal pairing on estimated probabilities of treatment and constrained imbalance on multiple variables. These tools will lead to reduced bias and improved precision by paying attention to aspects of the study's design. Third, tools for matched permutation inference will be extended to clustered observational studies with treatment given at both cluster and individual levels and possible spillover effects. An accompanying sensitivity analysis that addresses unobserved variables at both individual and cluster levels also will be constructed. Finally, a new measure for large-sample performance of weighting methods in the presence of unobserved variables will be constructed, quantifying the impact of design choices on robustness to bias from unobserved variables.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.