Discovering Fair Representations in the Data Domain
Discovering Fair Representations in the Data Domain
复制标题
DOI：
10.1109/cvpr.2019.00842
复制
发表时间：
2018-10
期刊：
影响因子：
--
通讯作者：
Novi Quadrianto;V. Sharmanska;Oliver Thomas
中科院分区：
文献类型：
--
作者：
Novi Quadrianto;V. Sharmanska;Oliver Thomas
可解释性和公平性在计算机视觉和机器学习应用中至关重要，特别是在处理人类结果时，例如根据可能包含照片的申请材料邀请或不邀请参加工作面试。实现公平的一个有希望的方向是通过学习数据表示来消除受保护特征的语义，从而能够减轻不公平的结果。然而，所有可用的模型都会学习潜在的嵌入，这是以不可解释为代价的。我们建议将此问题转化为数据到数据的转换，即学习从输入域到公平目标域的映射，其中强制执行公平性定义。这里的数据域可以是图像，或任何表格数据表示。如果我们有可用的公平目标数据，这项任务就会很简单，但事实并非如此。为了克服这个问题，我们通过利用残差统计数据（输入数据与其翻译版本之间的差异）和受保护的特征来学习高度不受约束的映射。当应用于以性别属性作为受保护特征的人脸图像 CelebA 数据集时，我们的模型通过调整眼睛和嘴唇区域来强制机会平等。有趣的是，在同一数据集上，当使用图像的语义属性表示进行翻译时，我们得出了类似的结论。在最近 DiF 数据集的面部图像上，具有相同的性别属性，我们的方法调整鼻子区域。在成人收入数据集中，同样具有受保护的性别属性，我们的模型通过混淆妻子和丈夫的关系等方式实现了机会平等。分析这些系统性变化将使我们能够仔细审查公平标准、选择的受保护特征和预测性能之间的相互作用。
Interpretability and fairness are critical in computer vision and machine learning applications, in particular when dealing with human outcomes, e.g. inviting or not inviting for a job interview based on application materials that may include photographs. One promising direction to achieve fairness is by learning data representations that remove the semantics of protected characteristics, and are therefore able to mitigate unfair outcomes. All available models however learn latent embeddings which comes at the cost of being uninterpretable. We propose to cast this problem as data-to-data translation, i.e. learning a mapping from an input domain to a fair target domain, where a fairness definition is being enforced. Here the data domain can be images, or any tabular data representation. This task would be straightforward if we had fair target data available, but this is not the case. To overcome this, we learn a highly unconstrained mapping by exploiting statistics of residuals -- the difference between input data and its translated version -- and the protected characteristics. When applied to the CelebA dataset of face images with gender attribute as the protected characteristic, our model enforces equality of opportunity by adjusting the eyes and lips regions. Intriguingly, on the same dataset we arrive at similar conclusions when using semantic attribute representations of images for translation. On face images of the recent DiF dataset, with the same gender attribute, our method adjusts nose regions. In the Adult income dataset, also with protected gender attribute, our model achieves equality of opportunity by, among others, obfuscating the wife and husband relationship. Analyzing those systematic changes will allow us to scrutinize the interplay of fairness criterion, chosen protected characteristics, and prediction performance.