Unpacking Compound Treatments in Email Audit Experiments

本研究项目将改进审计实验的设计和分析。审计实验是一种工具，用于测试在调查和访谈可能引起社会可取性偏见的情况下的歧视行为，并且已知会产生强烈的因果关系。然而，审计实验中的一个常见问题是干预与其他属性的潜在混淆。例如，研究人员经常用名字来表明种族，但名字也可能表明其他属性，如社会经济地位。使用收入或职业等其他信号作为额外的干预措施，可能有助于理清种族与社会经济地位的影响。本项目将开发改进的设计和方法，通过将潜在的混杂因素作为额外的干预措施纳入，从感兴趣的干预措施中分离出这些混杂因素的影响。将要制订的方法将用于研究种族和有关的社会经济因素对获得法律服务的影响。了解法律服务市场中的歧视很重要，因为获得律师服务是个人参与司法系统的关键第一步。该项目还将通过研究生的参与来支持审计研究的培训和参与。本研究计划将发展方法，以解决因具有析因结构的审计实验而导致的设计和分析中的复杂问题。这些方法将包括设计在资源限制下使信息最大化的最佳析因实验，开发用于评估干预措施因果效应的适当统计模型，以及与设计相一致的数据分析程序。现代因果推理理论和随机实验将成为新方法的基础。将开发分析方法，以评估干预措施对选定的一组受试者（有限人群）的因果影响，以及对更大的超级人群的因果影响，这些受试者可以假设是随机选择的。将开发无模型（基于随机化）和基于模型的因果效应评估方法。研究人员还将探索方法，以扩展设计和分析审计实验与二进制或两级干预的情况下，多层次的干预。新方法将应用于一项审计实验，以评估在回应法律服务请求时的种族歧视，使用电子邮件上的名字来表明种族。这项研究的结果将为研究法律上的种族歧视和诉诸司法系统的机会提供关键证据。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This research project will improve the design and analysis of audit experiments. Audit experiments are tools used to test for discriminatory behavior in situations where surveys and interviews can induce social desirability bias and are known to make strong causal claims. However, a common issue in audit experiments is the potential confounding of interventions with other attributes. For example, researchers often use names to signal race, but names may also signal other attributes, such as socio-economic status. Using other signals like income or occupation as additional interventions may help disentangle the effect of race from socio-economic status. This project will develop improved design and methodology for disentangling the effects of potential confounders from interventions of interest by incorporating such confounders as additional interventions. The methods to be developed will be applied to the study of the effects of race and related socio-economic factors on access to legal services. Understanding discrimination in the market for legal services is important because access to lawyers is a crucial first step for individuals to participate in the justice system. The project also will support training and engagement with audit studies through the involvement of graduate students.This research project will develop methods to address the complex problems in design and analysis that result from audit experiments having a factorial structure. These methods will include designing optimal factorial experiments that maximize information under resource constraints, development of appropriate statistical models for assessing causal effects of interventions, and data analysis procedures consistent with the design. The modern theory of causal inference and randomized experiments will serve as the foundation for the new methods. Analysis methods will be developed to assess causal effects of interventions on a selected group of subjects (finite population) as well as on a larger superpopulation from which such subjects can be assumed to be randomly selected. Methods for model-free (randomization-based) and model-based assessment of causal effects will be developed. The investigators also will explore methodology to extend design and analysis of audit experiments with binary or two-level interventions to the case of multilevel interventions. The new methods will be applied to an audit experiment to assess racial discrimination in response to requests for legal services, using names on emails to signal race. The results of this research will provide critical evidence for the study of racial discrimination in law and access to the justice system.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.