FAI: Measuring and Mitigating Biases in Generic Image Representations

视觉识别是人脑完成的一项了不起的任务。被训练来模拟这一能力的计算方法依赖于观察数百万个视觉输入与人类注释配对的例子。这些计算方法已经取得了很大的进步，并被越来越多地应用于许多面向用户的应用中，如图像搜索、自动图像标注、半自主导航系统、智能虚拟助手等。然而，这些系统中潜在的视觉识别模型经常通过将具有社会意义的敏感变量与其预测相关联而产生错误。该项目的目标是以系统的方式测量和减轻此类错误。例如，如果一种方法能够识别场景的图像，例如“教室”，则该项目的目标是确保根据诸如白板、椅子、书桌和空间用作教室通常所需的其他元素的存在等线索来获得这样的预测，而不是基于诸如教室中人的特征或属性之类的附带元素。为此，本项目旨在更容易地确定计算视觉识别的方法在多大程度上依赖于与附带元素的虚假关联。本项目将研究当前计算视觉识别的方法和模型中存在的社会偏见，这些方法和模型被广泛用作一般视觉表征的来源。开发的方法将基于来自机器学习、计算机视觉和软件测试社区的坚实基础。该项目引入了联想测试来探测在各种条件下训练的模型，以系统地消除在一般视觉表征学习过程中引入的偏差。该项目将1)开发一种通用评估方法来测量通用视觉表征学习中的各种类型的偏差，2)提出减少这些偏差在现有通用视觉表征提取模型中的影响的方法，以及3)测量这些偏差对一些关键下游任务的影响。这三个研究目标将由一项全面的评价计划和更广泛的参与活动加以补充。这项研究工作将为计算机视觉模型和方法的预测带来新的见解，以对此类模型的部署中的风险做出明智的决定。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Visual recognition is a remarkable task performed by the human brain. Computational methods trained to emulate this capability rely on observing millions of examples of visual input paired with human annotations. These computational methods have made great progress and are being increasingly adopted in many user-facing applications such as image search, automated image tagging, semi-autonomous navigation systems, smart virtual assistants, etc. However, the underlying visual recognition models in these systems often produce errors by associating sensitive variables of societal significance with their predictions. The goal of this project is to measure and mitigate such errors in a systematic fashion. For example, if a method is able to recognize images of scenes such as 'classroom', the goal of this project is to ensure that such predictions are obtained based on cues such as the presence of a whiteboard, chairs, desks, and other elements typically needed for a space to function as a classroom and not based on incidental elements such as the characteristics or attributes of people present in the classroom. To this end, this project aims to make it easier to determine to what extent methods for computational visual recognition rely on spurious associations with incidental elements.This project will provide a study of societal biases present in current methods and models for computational visual recognition that are widely used as a source of generic visual representations. The developed methods will be based on solid foundations drawn from both the machine learning, computer vision, and software testing communities. The project introduces association tests to probe models trained under a variety of conditions to systematically disentangle the biases introduced during generic visual representation learning. The project will be 1) developing a general assessment methodology to measure various types of biases in generic visual representation learning, 2) proposing methods to diminish the impact of these biases in existing generic visual representation extraction models, and 3) measuring the impact of these biases on some key downstream tasks. These three research aims will be complemented by a comprehensive evaluation plan and broadening participation activities. This research effort will bring novel insights into the sources of biases in the predictions of computer vision models and methodologies to make informed decisions about the risks in the deployment of such models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.