Human-assisted computer vision

计算机视觉的一个长期目标是创造一种软件，可以从图像中解释场景内容，其性能与人类视觉系统相似，甚至更好。我的研究一直致力于估计场景属性，如表面的3-D布局和形状，它们的材料属性，如颜色和反射率，以及场景照明。传统的目标一直是设计尽可能自主的计算机视觉算法，特别是尽可能少地依赖人类帮助来约束解决方案。许多具体的计算机视觉问题确实可以在很少的人工帮助下得到解决，尽管仍然有许多重要的问题无法解决。这项拟议的研究将解决一个新的、不同的目标，即通过允许比过去使用的更高水平的人类协助来实现更好的计算机视觉性能。这种用户参与适合于许多任务，这些任务不需要自治系统，并且目标是尽可能准确地解决问题。将讨论基本研究问题，例如需要什么类型和多少人工辅助才能达到给定水平的计算机视觉性能。这项研究将开发出将人类和计算机视觉相结合的新方法。我们将使用现有的心理物理方法，即形状探测器，来测量人类感知的场景几何形状，并将这些感知测量作为计算机视觉算法求解的约束。这是进行计算机视觉的一种显著不同的方法。我们不会问在没有(或最少)人工辅助的情况下可以达到什么性能水平，而是打开问题，研究各种级别和类型的人工辅助可以在多大程度上提高算法的性能。我们将研究的视觉问题非常具有挑战性，包括对树木和灌木丛等非常杂乱的场景进行3D立体重建，以及使用组合焦点和纹理线索在自然场景中进行3D形状和布局重建。

A longstanding goal of computer vision has been to create software that can interpret scene content from images with a performance that is similar or even better than the human visual system. My research has been concerned with estimating scene properties such as the 3-D layout and shapes of surfaces, their material properties such as color and reflectance, and the scene illumination. The traditional goal has been to design computer vision algorithms that are as autonomous as possible, in particular, that rely as little as possible on human assistance to constrain the solution. Many specific computer vision problems can indeed can be solved with very little human assistance, though there remain many important problems that cannot. The proposed research will address a new and different goal, which is to achieve better computer vision performance by allowing for greater levels of human assistance than have been used in the past. Such user involvement is appropriate for many tasks, that do not require an autonomous system and for which the goal is as accurate solution as possible. Basic research questions will be addressed, such as what type and how much human assistance is required to achieve a given level of computer vision performance. The research will develop new methods that combine human and computer vision. We will use established psychophysical methods, namely shape probes, to measure the scene geometry that is perceived by humans, and we will apply these perceptual measurements as constraints on the solution of the computer vision algorithms. This is a significantly different approach to doing computer vision. Rather than asking what performance level can be attained using no (or minimal) human assistance, we will open up the problem and examine how much various levels and types of human assistance can improve the performance of the algorithms. The vision problems we will examine are very challenging, and include 3D stereo reconstruction of very cluttered scenes such as trees and bushes, as well as 3D shape and layout reconstruction in natural scenes using combined focus and texture cues.