Causal and Counterfactual Reasoning for Fairness

一家机构是否出于公平的理由做出决定，是当今世界越来越需要解决的问题。招聘员工是基于他们的才能而不是他们的性别吗？大学录取是基于申请者的学术能力，还是基于他们的教育/社会经济背景？司法系统是否仅仅根据种族对人们进行了更严厉的惩罚？这样的问题已经被问了很多年，现在，在一个算法为这些问题提供建议的世界里，拥有可靠的统计方法来测试偏见的证据就更重要了，这些方法不依赖于能够向决策者询问特定决策背后的原因。该项目的最初目标将是开发使用因果推理和反事实推理的测试，以期研究我们如何构建做出公平决定的算法。因果推理领域可以为我们提供思考公平问题的原则性工具，并将形成该项目的关键基础。因果推理使我们能够在数学上正式化，并在某些情况下回答反事实的问题，例如：如果性别不同，某人获得工作面试的机会会改变吗？这样的问题构成了一些简单的、非统计测试的基础，这些测试用于公平。例如，一个典型的测试就业市场中的偏见的方法是发送多份简历，其中两份简历除了名字之外都是一模一样的，其中一个名字表明这是一名男性的简历，另一份是女性的简历。如果一份简历收到的面试申请数量与另一份截然不同，我们会确信这是由于招聘程序中的某种偏见造成的。但这些研究试图回答我们刚才提出的完全相反的问题：如果性别不同，某人获得工作面试的机会会发生变化吗？这个例子展示了这些反事实的问题是如何抓住公平的直观定义的。现在假设对于上面的例子，我们要做一个简单的实验，并收集关于每个简历的回复数量的数据，我们需要一些方法来量化回复数量是否“非常不同”。要做到这一点，我们将使用统计假设检验，其目的是看看是否有证据表明差异是显著的，或者是否仅仅是偶然的？我们在这里试图回答的问题是，是否有证据表明，在这两份简历下，回答的分布是不同的。在这个项目中，目标是使用内核方法来生成测试，以查看数据是否显示出在我们将使用因果框架提出的反事实问题下分布不同的证据。我们使用核方法，因为它们允许我们在更复杂的环境中测试这个问题，而不需要对潜在的分布做出任何假设。这项研究的大部分将是这些测试的理论形成。然而，对它们进行计算测试以确保它们正常工作也是很重要的。这可以在真实世界的数据集上完成，或者通过模拟数据来模拟偏差，然后看看测试是否能检测到这一点。在实验之后，发布代码库可能是有益的，以便这些测试可以在更广泛的世界中使用，以检查不公平的决策并促进总体上更公平的决策。这个项目将属于EPSRC的“统计和应用概率”研究领域。

The question of whether an institution makes decisions for fair reasons is one that increasingly needs to be tackled in the world today. Are people being hired for jobs on the basis of their talents not their gender? Are university admissions based off applicant's academic ability, or their schooling/socio-economic background? Does the judicial system punish people more harshly solely on the basis of their race? Questions like this have been asked for many years, and now in a world where algorithms make recommendations for these questions it is even more important to have sound statistical methods to test for evidence of bias that do not rely on being able to ask the decision-maker for the reasons behind particular decisions. The initial aim of the project will be to develop such tests using causal and counterfactual reasoning with a view to researching how we can construct algorithms that will make fair decision.The field of causal inference can provide us with principled tools to think about questions of fairness and will form a key foundation for the project. Causal inference allows us to mathematically formalise and, in some cases, answer counterfactual questions such as: Would someone's chances of getting a job interview change if their gender was different? Questions such as this form the basis for some simple, non-statistical tests that are used for fairness. For example, a classic test for bias in the job market would be to send out multiple applications with two CVs that are identical apart from the name, with one name suggesting it is the CV of a man and one a woman. If a vastly different number of interview requests are received for one CV over the other, we would feel confident that this is due to some kind of bias in the hiring procedure. But these studies are trying to get at exactly the same counterfactual question we have just posed, would someone's chances of getting a job interview change if their gender was different? This example demonstrates how these counterfactual questions capture an intuitive definition of fairness. Now suppose for the above example we were to do the simple experiment and collect data on the number of responses for each CV, we would need some way to quantify if the number of responses is "vastly different". To do this, we would use a statistical hypothesis test which aims to see if there is evidence that the difference is notable, or if it could just be due to chance? The question we are trying to answer here is if there is evidence that the distribution of the responses is different under the two CVs. In this project the aim will be to use kernel methods to produce tests to see if data shows evidence the distribution is different under the counterfactual questions we will pose using the causal framework. We use kernel methods as they allow us to test this question in more complex settings without making any assumptions about the underlying distributions.The majority of this research will be the theoretical formation of these tests. However, it will also be important to test them computationally to ensure they work correctly. This could be done on real world data sets or by simulating data to emulate bias and then see if the tests can pick up on this. Following the experiments, it may be beneficial to release the code base in order that these tests can be used in the wider world to check for unfair decision making and to promote fairer decision making generally. This project will fall under the EPSRC Research Area of 'Statistics and Applied Probability'.