FAI: Measuring and Mitigating Biases in Generic Image Representations

视觉识别是人类大脑完成的一项了不起的任务。训练来模拟这种能力的计算方法依赖于观察数百万与人类注释配对的视觉输入示例。这些计算方法已经取得了很大的进步，并越来越多地应用于许多面向用户的应用，如图像搜索、自动图像标记、半自动导航系统、智能虚拟助手等。然而，这些系统中潜在的视觉识别模型经常通过将社会意义的敏感变量与其预测相关联而产生错误。这个项目的目标是以系统的方式度量和减轻这些错误。例如，如果一种方法能够识别像“教室”这样的场景图像，那么这个项目的目标是确保这种预测是基于诸如白板、椅子、桌子和其他空间作为教室通常需要的元素的存在等线索来获得的，而不是基于诸如教室中出现的人的特征或属性等偶然元素。为此，本项目旨在使计算视觉识别方法在多大程度上依赖于附带元素的虚假关联变得更容易。该项目将研究当前计算视觉识别方法和模型中存在的社会偏见，这些方法和模型被广泛用作通用视觉表示的来源。开发的方法将基于从机器学习、计算机视觉和软件测试社区汲取的坚实基础。该项目引入了关联测试来探测在各种条件下训练的模型，以系统地解开通用视觉表征学习过程中引入的偏见。该项目将包括：1)开发一种通用评估方法来衡量通用视觉表示学习中各种类型的偏差；2)提出减少这些偏差在现有通用视觉表示提取模型中的影响的方法；3)测量这些偏差对一些关键下游任务的影响。这三个研究目标将由一项综合评价计划和扩大参与活动加以补充。这项研究将为计算机视觉模型和方法预测中的偏差来源带来新的见解，从而对部署此类模型的风险做出明智的决策。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Visual recognition is a remarkable task performed by the human brain. Computational methods trained to emulate this capability rely on observing millions of examples of visual input paired with human annotations. These computational methods have made great progress and are being increasingly adopted in many user-facing applications such as image search, automated image tagging, semi-autonomous navigation systems, smart virtual assistants, etc. However, the underlying visual recognition models in these systems often produce errors by associating sensitive variables of societal significance with their predictions. The goal of this project is to measure and mitigate such errors in a systematic fashion. For example, if a method is able to recognize images of scenes such as 'classroom', the goal of this project is to ensure that such predictions are obtained based on cues such as the presence of a whiteboard, chairs, desks, and other elements typically needed for a space to function as a classroom and not based on incidental elements such as the characteristics or attributes of people present in the classroom. To this end, this project aims to make it easier to determine to what extent methods for computational visual recognition rely on spurious associations with incidental elements.This project will provide a study of societal biases present in current methods and models for computational visual recognition that are widely used as a source of generic visual representations. The developed methods will be based on solid foundations drawn from both the machine learning, computer vision, and software testing communities. The project introduces association tests to probe models trained under a variety of conditions to systematically disentangle the biases introduced during generic visual representation learning. The project will be 1) developing a general assessment methodology to measure various types of biases in generic visual representation learning, 2) proposing methods to diminish the impact of these biases in existing generic visual representation extraction models, and 3) measuring the impact of these biases on some key downstream tasks. These three research aims will be complemented by a comprehensive evaluation plan and broadening participation activities. This research effort will bring novel insights into the sources of biases in the predictions of computer vision models and methodologies to make informed decisions about the risks in the deployment of such models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.