Human-assisted computer vision

计算机视觉的一个长期目标是创造出能够从图像中解读场景内容的软件，其性能与人类视觉系统相似甚至更好。我的研究一直关注于估计场景属性，如三维布局和表面形状，它们的材料属性，如颜色和反射率，以及场景照明。传统的目标是设计尽可能自主的计算机视觉算法，特别是尽可能少地依赖于人类的帮助来约束解决方案。许多特定的计算机视觉问题确实可以在很少的人工帮助下解决，尽管仍然有许多重要的问题不能解决。拟议的研究将解决一个新的和不同的目标，即通过允许比过去使用的更高水平的人类协助来实现更好的计算机视觉性能。这种用户参与适合于许多任务，这些任务不需要自治系统，并且目标是尽可能准确地解决问题。基础研究问题将被解决，比如什么类型和多少人的协助是需要达到给定水平的计算机视觉性能。这项研究将开发结合人类和计算机视觉的新方法。我们将使用已建立的心理物理学方法，即形状探针，来测量人类感知的场景几何，并将这些感知测量作为计算机视觉算法解决方案的约束。这是一种非常不同的计算机视觉方法。我们不会问在没有（或最少）人工帮助的情况下可以达到什么样的性能水平，而是会打开这个问题，并检查不同级别和类型的人工帮助可以在多大程度上提高算法的性能。我们将研究的视觉问题是非常具有挑战性的，包括非常杂乱的场景，如树木和灌木的3D立体重建，以及3D形状和布局重建在自然场景使用组合焦点和纹理线索。

A longstanding goal of computer vision has been to create software that can interpret scene content from images with a performance that is similar or even better than the human visual system. My research has been concerned with estimating scene properties such as the 3-D layout and shapes of surfaces, their material properties such as color and reflectance, and the scene illumination. The traditional goal has been to design computer vision algorithms that are as autonomous as possible, in particular, that rely as little as possible on human assistance to constrain the solution. Many specific computer vision problems can indeed can be solved with very little human assistance, though there remain many important problems that cannot. The proposed research will address a new and different goal, which is to achieve better computer vision performance by allowing for greater levels of human assistance than have been used in the past. Such user involvement is appropriate for many tasks, that do not require an autonomous system and for which the goal is as accurate solution as possible. Basic research questions will be addressed, such as what type and how much human assistance is required to achieve a given level of computer vision performance. The research will develop new methods that combine human and computer vision. We will use established psychophysical methods, namely shape probes, to measure the scene geometry that is perceived by humans, and we will apply these perceptual measurements as constraints on the solution of the computer vision algorithms. This is a significantly different approach to doing computer vision. Rather than asking what performance level can be attained using no (or minimal) human assistance, we will open up the problem and examine how much various levels and types of human assistance can improve the performance of the algorithms. The vision problems we will examine are very challenging, and include 3D stereo reconstruction of very cluttered scenes such as trees and bushes, as well as 3D shape and layout reconstruction in natural scenes using combined focus and texture cues.