Human-assisted computer vision

点击翻译按钮获取中文摘要

A longstanding goal of computer vision has been to create software that can interpret scene content from images with a performance that is similar or even better than the human visual system. My research has been concerned with estimating scene properties such as the 3-D layout and shapes of surfaces, their material properties such as color and reflectance, and the scene illumination. The traditional goal has been to design computer vision algorithms that are as autonomous as possible, in particular, that rely as little as possible on human assistance to constrain the solution. Many specific computer vision problems can indeed can be solved with very little human assistance, though there remain many important problems that cannot. The proposed research will address a new and different goal, which is to achieve better computer vision performance by allowing for greater levels of human assistance than have been used in the past. Such user involvement is appropriate for many tasks, that do not require an autonomous system and for which the goal is as accurate solution as possible. Basic research questions will be addressed, such as what type and how much human assistance is required to achieve a given level of computer vision performance. The research will develop new methods that combine human and computer vision. We will use established psychophysical methods, namely shape probes, to measure the scene geometry that is perceived by humans, and we will apply these perceptual measurements as constraints on the solution of the computer vision algorithms. This is a significantly different approach to doing computer vision. Rather than asking what performance level can be attained using no (or minimal) human assistance, we will open up the problem and examine how much various levels and types of human assistance can improve the performance of the algorithms. The vision problems we will examine are very challenging, and include 3D stereo reconstruction of very cluttered scenes such as trees and bushes, as well as 3D shape and layout reconstruction in natural scenes using combined focus and texture cues.