RUI: Ground-Truth Driven Stereo Algorithm Design

该项目旨在促进对基于能量的立体匹配技术的理解。使用新的立体数据集与地面真实差异，研究将解决这个问题：“什么是正确的能量函数最小化？”具体目标包括开发一个自动化系统，用于获取具有地面真实度的高精度多视图立体数据集，并设计新的能量函数，其最小化产生的差异(1)准确地模拟遮挡，(2)使用单眼线索将遮挡边界与物体边界对齐，以及(3)在半遮挡区域创建合理的视差假设。在这些具有挑战性的任务中表现良好的立体算法在新兴的消费者级应用中有着重要的应用，包括用于远程会议的虚拟凝视校正，以及用于3D可视化的基于图像的对象模型的创建。研究还将侧重于开发快速、近似的能量最小化技术，包括图切和动态规划方法的变体。本科生将积极参与这项研究的所有组成部分，特别是在数据采集和测试阶段。该项目的智力优势在于为计算机视觉社区提供高质量的、具有地面真实性的多视图数据集，并提高基于能量的图像匹配技术的最新水平。该项目的更广泛影响包括提高计算机视觉方法在电信和电子商务等日益成为社会核心的应用中的适用性；也有机会让文理学院的本科生接触到研究、实验和发现的世界。

This project aims to advance the understanding of energy-based stereo-matching techniques. Using new stereo data sets with ground-truth disparities, the research will address the question: "What is the right energy function to minimize?" The specific goals include developing an automated system for obtaining highly accurate multi-view stereo data sets with ground truth, and designing new energy functions whose minimization yields disparities that (1) accurately model occlusion, (2) align occlusion boundaries with object boundaries using monocular cues, and (3) create reasonable disparity hypotheses in half-occluded regions. Stereo algorithms that perform well on these challenging tasks have important applications in emerging consumer-level applications, including virtual gaze correction for teleconferencing, and the creation of image-based object models for 3D visualization. The research will also focus on developing fast, approximate energy minimization techniques, including variants of graph-cut and dynamic programming methods. Undergraduate students will be actively involved in all components of this research, in particular in the data acquisition and testing stages. The intellectual merits of the project are to provide the computer vision community with high-quality, multi-view data sets with ground truth and to improve the state of the art in energy-based image-matching techniques. The broader impacts of the project include the improved applicability of computer vision methods to applications that are becoming central to society, such as telecommunication and e-commerce; and the opportunity to expose undergraduates at a liberal-arts college to the world of research, experimentation, and discovery.