CAREER: Holistic Scene Understanding with Multiple Hypotheses from Vision Modules

这个项目开发了从图像中整体理解场景的算法和技术。构建下一代视觉系统的关键障碍是模糊性。例如，图像中的面片可能看起来像一张脸，但可能只是树枝和阴影的附带排列。因此，一个单独运行的视觉模块经常会产生毫无意义的结果，比如产生漂浮在稀薄空气中的幻觉脸。该项目开发了一个视觉系统，该系统可以从不同的视觉模块，如3D场景布局、物体布局和姿态估计，联合推理多个似然假设。开发的技术有可能改善视觉系统并产生根本性影响--从无人驾驶汽车为身体残疾的人带来移动性，到帮助执法部门在灾难中进行搜索和救援的无人机。该项目涉及与教育和推广紧密结合的研究，以培养下一代年轻科学家和研究人员。这项研究通过从计算机视觉模块中提取和利用一小部分不同的似是而非的假设或猜测(例如，一个补丁可能是{天空或垂直表面}x{脸或树枝})来解决联合推理中的基本挑战。该项目产生了新的知识和技术，用于(1)从不同的视觉模块生成一小部分不同的可信假设，(2)在所有模块上进行联合推理，从每个模块中挑选一个单独的假设，以及(3)通过主动征求用户仅对一小部分可信假设的反馈来减少人工注释工作。项目网页：http://computing.ece.vt.edu/~dbatra

This project develops algorithms and techniques for holistic scene understanding from images. The key barrier to building the next generation of vision systems is ambiguity. For example, a patch from an image may look like a face but may simply be an incidental arrangement of tree branches and shadows. Thus, a vision module operating in isolation often produces nonsensical results, such as hallucinating faces floating in thin air. This project develops a visual system that jointly reasons about multiple plausible hypotheses from different vision modules such as 3D scene layout, object layout, and pose estimation. The developed technologies have the potential to improve vision systems and make fundamental impact - from self-driving cars bringing mobility to the physically impaired, to unmanned aircrafts helping law enforcement with search and rescue in disasters. The project involves research tightly integrated with education and outreach to train the next generation of young scientists and researchers. This research addresses the fundamental challenge in joint reasoning by extracting and leveraging a small set of diverse plausible hypotheses or guesses from computer vision modules (e.g. a patch may be a {sky or a vertical surface} x {face or tree branches}). This project generates new knowledge and techniques for (1) generating a small set of diverse plausible hypotheses from different vision modules, (2) joint reasoning over all modules to pick a single hypothesis from each module, and (3) reducing human annotation effort by actively soliciting user feedback only on the small set of plausible hypotheses. Project Webpage: http://computing.ece.vt.edu/~dbatra