RI: SMALL: Recognizing objects in images and their properties over time

计算机视觉生活在数据集的黄金时代。人类视觉的所有方面都被系统地绘制并转录成一个越来越大的标签数据池。所有人都有一个单一的目标：教会一个视觉系统，现在是一个深度网络，来模仿人类感知的所有方面。目前的方法很简单：收集足够的标签数据，然后使用有监督的机器学习来模仿监督。这种方法只有一个问题：以这种方式训练的系统仅限于模拟单一的狭隘任务。在这个项目中，我们朝着将许多视觉任务统一到一个单一系统中迈出了一步：一个通过时间推断所有事物的所有属性的框架。如果成功，该系统可以识别任何新的未知图像中的对象及其所有属性，并将计算机视觉的全部力量带给非专家。应用包括通过操纵物体与世界互动的自主代理，以及通过移动物体及其属性观察世界的老年人辅助技术。该项目将进行三项研究。1.所有对象的检测：在对象检测中，数据集专门针对领域。驾驶数据集描述了任何可以想象到的车辆类型，室内数据集专注于常见的家庭物品，行人数据集专门关注人类。我们如何训练一个利用所有这些数据源的物体检测系统？我们如何才能使这些不同的数据源相互关联呢？如何处理某些数据源中的部分注释？2.推断所有属性：对象检测构成了视觉推理许多方面的基本构建块。然而，最有趣的任务是在检测之后开始的：物体的2D或3D姿势是什么？这个物体是可变形的吗？这会对自动驾驶汽车构成危险吗？同样，有数百个任务和数据源描述对象的所有属性。我们如何才能学会一个能推断出它们全部的探测器呢？3.通过时间进行识别：最后，探测不应该在时间上孤立。我们如何通过时间对物体和属性进行推理？我们能学会使用当前基于图像的数据集在时间上连贯的方式识别物体吗？这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Computer vision lives in the golden age of datasets. All aspects of human vision are systematically mapped and transcribed into an ever-larger pool of labeled data. All with one single goal: teach a vision system, nowadays a deep network, to imitate all aspects of human perception. The current recipe is simple: collect sufficient labeled data, then use supervised machine learning to mimic the supervision. There is just one issue with this approach: Systems trained this way are limited to imitate a single narrow task. In this project, we take a step towards unifying many vision tasks into one single system: A framework that infers all properties of all things through time. If successful, this system can identify objects and all their properties in any new unseen image, and bring the full power of computer vision to the non-expert. Applications include autonomous agents interacting with the world through the manipulation of objects, and assistive technologies for the elderly that observes the world through moving objects and their properties.The project will pursue three research thrusts. 1. Detecting all objects: In object detection, datasets specialize in domains. Driving datasets describe any vehicle type imaginable, indoor datasets focus on common household objects, and pedestrian datasets exclusively focus on humans. How can we train an object detection system that leverages all these sources of data? How can we relate these different data sources to each other? How do we deal with partial annotation in some data sources? 2. Inferring all properties: Object detection forms the basic building block for many aspects of visual reasoning. However, the most interesting tasks start after detection: What is the 2D or 3D pose of an object? Is this object deformable? Could it be a danger to an autonomous vehicle? Again, there are hundreds of tasks and data sources that describe all the properties of objects. How can we learn a detector that infers them all? 3. Recognition through time: Finally, detection should not be isolated in time. How do we reason about objects and properties through time? Can we learn to recognize objects in a temporally coherent manner using current image-based datasets?This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.