CAREER: Holistic Scene Understanding with Multiple Hypotheses from Vision Modules

该项目开发了从图像中理解整体场景的算法和技术。构建下一代视觉系统的关键障碍是模糊性。例如，来自图像的补丁可能看起来像一张脸，但可能只是树枝和阴影的偶然排列。因此，孤立运行的视觉模块通常会产生无意义的结果，例如漂浮在稀薄空气中的幻觉。本项目开发了一个视觉系统，该系统联合推理来自不同视觉模块的多个合理假设，如3D场景布局，对象布局和姿态估计。所开发的技术有可能改善视觉系统并产生根本性的影响-从自动驾驶汽车为身体受损者带来移动性，到无人驾驶飞机帮助执法部门在灾难中进行搜索和救援。该项目涉及研究与教育和外联紧密结合，以培训下一代年轻科学家和研究人员。 这项研究通过从计算机视觉模块中提取和利用一小部分不同的合理假设或猜测来解决联合推理中的基本挑战（例如，一个补丁可能是{天空或垂直表面} x {人脸或树枝}）。该项目产生了新的知识和技术，用于（1）从不同的视觉模块中生成一小部分不同的合理假设，（2）对所有模块进行联合推理，从每个模块中选择一个假设，以及（3）通过主动征求用户对一小部分合理假设的反馈来减少人工注释工作。项目网页：http://computing.ece.vt.edu/~dbatra

This project develops algorithms and techniques for holistic scene understanding from images. The key barrier to building the next generation of vision systems is ambiguity. For example, a patch from an image may look like a face but may simply be an incidental arrangement of tree branches and shadows. Thus, a vision module operating in isolation often produces nonsensical results, such as hallucinating faces floating in thin air. This project develops a visual system that jointly reasons about multiple plausible hypotheses from different vision modules such as 3D scene layout, object layout, and pose estimation. The developed technologies have the potential to improve vision systems and make fundamental impact - from self-driving cars bringing mobility to the physically impaired, to unmanned aircrafts helping law enforcement with search and rescue in disasters. The project involves research tightly integrated with education and outreach to train the next generation of young scientists and researchers. This research addresses the fundamental challenge in joint reasoning by extracting and leveraging a small set of diverse plausible hypotheses or guesses from computer vision modules (e.g. a patch may be a {sky or a vertical surface} x {face or tree branches}). This project generates new knowledge and techniques for (1) generating a small set of diverse plausible hypotheses from different vision modules, (2) joint reasoning over all modules to pick a single hypothesis from each module, and (3) reducing human annotation effort by actively soliciting user feedback only on the small set of plausible hypotheses. Project Webpage: http://computing.ece.vt.edu/~dbatra