CAREER: Holistic Scene Understanding with Multiple Hypotheses from Vision Modules

该项目开发了从图像中理解整体场景的算法和技术。构建下一代视觉系统的主要障碍是模糊性。例如，图像中的一个补丁可能看起来像一张脸，但可能只是树枝和阴影的偶然排列。因此，孤立运行的视觉模块通常会产生荒谬的结果，例如幻觉中的面孔漂浮在空气中。该项目开发了一个视觉系统，该系统可以从不同的视觉模块（如3D场景布局，物体布局和姿态估计）共同推理多个合理的假设。这些已开发的技术有可能改善视觉系统，并产生根本性的影响——从为身体受损的人带来行动能力的自动驾驶汽车，到在灾难中帮助执法部门进行搜索和救援的无人驾驶飞机。该项目涉及的研究与教育和外联紧密结合，以培养下一代年轻科学家和研究人员。本研究通过从计算机视觉模块中提取和利用一组不同的貌似合理的假设或猜测（例如，一个补丁可能是{天空或垂直表面}x{脸或树枝}）来解决联合推理中的基本挑战。该项目产生了新的知识和技术：(1)从不同的视觉模块生成一组不同的合理假设，(2)对所有模块进行联合推理，从每个模块中选择一个假设，以及(3)通过积极征求用户对一小组合理假设的反馈来减少人工注释的工作量。项目网页：http://computing.ece.vt.edu/~dbatra

This project develops algorithms and techniques for holistic scene understanding from images. The key barrier to building the next generation of vision systems is ambiguity. For example, a patch from an image may look like a face but may simply be an incidental arrangement of tree branches and shadows. Thus, a vision module operating in isolation often produces nonsensical results, such as hallucinating faces floating in thin air. This project develops a visual system that jointly reasons about multiple plausible hypotheses from different vision modules such as 3D scene layout, object layout, and pose estimation. The developed technologies have the potential to improve vision systems and make fundamental impact - from self-driving cars bringing mobility to the physically impaired, to unmanned aircrafts helping law enforcement with search and rescue in disasters. The project involves research tightly integrated with education and outreach to train the next generation of young scientists and researchers. This research addresses the fundamental challenge in joint reasoning by extracting and leveraging a small set of diverse plausible hypotheses or guesses from computer vision modules (e.g. a patch may be a {sky or a vertical surface} x {face or tree branches}). This project generates new knowledge and techniques for (1) generating a small set of diverse plausible hypotheses from different vision modules, (2) joint reasoning over all modules to pick a single hypothesis from each module, and (3) reducing human annotation effort by actively soliciting user feedback only on the small set of plausible hypotheses. Project Webpage: http://computing.ece.vt.edu/~dbatra