EAGER: Quantifying and Reducing Data Bias in Object Detection Using Physics-based Image Synthesis

这个项目开发了改进的计算机视觉方法，用于自动识别现实环境中图像中的任意对象。对象识别通常通过拟合将图像映射到可能的对象位置和标签的函数来执行。这样的函数被安装(训练)在示例图像及其人类分配的对象位置和标签的数据库上。这项研究可以为与社会相关的应用程序带来更准确的视觉感知，例如执行家务、帮助老年人、应对灾难和快速学习新的制造和服务技能的机器人。它还可以为更广泛的社区提供一个共同的代码库，为领域适应问题提供新的数据集挑战，传播科学技术成果和相关的课件，并进行具体的外联，以确保代表不足的群体的广泛参与。第一个目标是确定数据集中潜在物理因素的覆盖范围，以满足人类在任意领域的表现所需。这项研究包括现有的数据集和使用不同程度的照片真实感的图形渲染技术生成的新数据集。目标是发展一种关于给定数据集的物理复杂性的理论，以及相对于给定的图像表示和学习框架，它如何影响对现实世界对象识别任务的泛化。物理参数包括但不限于：3D形状、表面颜色、纹理、背景/场景、相机视点、传感器噪声、照明、镜面反射和投射阴影。第二个研究目标是学习不随数据偏差的某些物理原因而变化的图像表征。其目标是开发能够从真实和非照片真实合成数据的组合中学习的模型和表示学习方法，并且能够抵抗常见的数据偏向来源。这些表示包括简单的基于边的描述符，以及更一般的基于卷积和池化运算层的分层表示。

This project develops improved computer vision methods for automatic recognition of arbitrary objects in images from realistic environments. Object recognition is typically performed by fitting a function that maps an image to likely object locations and labels. Such a function is fitted (trained) on a database of example images along with their human-assigned object locations and labels. This research can result in more accurate visual perception for socially relevant applications, such as robots performing household tasks, assisting the elderly, responding to disasters and quickly learning new manufacturing and service skills. It can also provide a common codebase for the wider community, new dataset challenges for domain adaptation problems, the dissemination of scientific and technical results and associated courseware, and specific outreach to ensure broad participation of underrepresented groups.The specific research agenda is structured around two aims. The first aim is to establish bounds on the coverage of latent physical factors in datasets needed for human-level performance on arbitrary domains. The study involves both existing datasets and new datasets generated using graphics rendering techniques at various degrees of photorealism. The goal is to develop a theory of the physical complexity of a given dataset and how it affects generalization to real world object recognition tasks, with respect to a given image representation and learning framework. Physical parameters include but are not limited to: 3D shape, surface color, texture, background/scene, camera viewpoint, sensor noise, lighting, specularities and cast shadows. The second research aim is to learn image representations invariant to some of the physical causes of data bias. The goal is to develop model and representation learning methods that are able to learn from a combination of real and non-photorealistic synthetic data, and are resistant to common sources of data bias. The representations include simple edge-based descriptors, and more generally hierarchical representations based on layers of convolution and pooling operations.