Estimating the intrinsic characteristics of real images to aid analysis

人类发现看东西毫不费力，这掩盖了一个事实，即理解视觉世界是一个非常困难的问题。视觉是困难的，因为我们看到的每一张图像都可能是由无数的物体和光线组合而成的。想想最简单的图像属性——灰度。图像中每个像素的灰度是由落在每个物体上的光量和从每个物体反射回来的光量决定的。深色物体的灰度值比浅色物体低，但即使是浅色物体在阴影中也有较低的灰度值。我们无法通过测量灰度来判断我们是在明亮的光线下看黑暗的物体还是在阴影下看明亮的物体。更糟糕的是，当图像的不同部分的灰度水平不同时，我们无法判断这种差异是由于有两个物体还是由于光量的变化。尽管存在这个问题，人类还是很擅长找出灰阶变化的原因；我们能从阴影中分辨出物体。我们如此善于理解图像中发生的事情的一个原因可能是我们使用其他属性，如颜色和图案来告诉我们灰度的含义。这个想法导致了“内在形象”的概念。内在图像是描述场景的一个属性的图像。因此，与其将阴影和物体反射率混合在一起，我们不如制作两幅内在图像，分别用于阴影和反射率。科学家们已经成功地利用颜色变化来计算出灰度的含义，从而产生了这样的内在图像。但是，阴影和反射的类型不止一种。我们想改进现有的方法，产生四张而不是两张内在图像。我们的第一个内在图像将包含在起伏表面上发现的柔和阴影类型。我们的第二个固有图像将包含坚硬的阴影，当物体阻挡光线时产生。我们的第三个本征图像将描述哑光物体的反射率，而我们的第四个图像将描述来自闪亮物体的反射。为了区分这四幅图像，我们需要使用除颜色以外的其他信息。我们认为表面图案（如木纹）将提供必要的信息。对于那些试图让计算机理解图像中发生的事情的工程师来说，提取四张内在图像将非常有帮助。仅举一个例子，人类似乎非常擅长从阴影的方式来估计表面上起伏的形状。我们在这方面做得很好，以至于我们会自动做到这一点，而编写计算机软件的人可以通过在边缘添加一些高光和阴影来欺骗我们，让我们认为他们的“按钮”从屏幕中脱颖而出。有许多计算机程序试图通过阴影来解释形状。虽然许多这些程序工作得很好，但它们倾向于假设所有灰色级别的变化都是由于阴影，这是由于表面波动造成的。我们知道，这种假设在真实的图片中是不成立的，这些程序在看到这样的图片时往往表现不佳。但是如果我们可以从真实的图像中生成阴影图像，那么这些程序可能会更好地工作。为了测试我们的想法并决定实现预期结果的最佳方式，我们将收集大量物体的照片，这些物体的形状我们要么已经知道，要么可以计算出来。我们将非常仔细地校准这些图片，然后使用它们来锻炼颜色和图案所传达的信息，这些信息可以帮助我们锻炼每个灰度变化的含义。我们还将测试人类，看看他们可能会使用哪些线索。我们将把我们的图片放到互联网上，这样其他人也可以尝试他们的想法。我们打算与一家软件公司合作，他们将采纳我们最好的想法，并将它们实现在一个计算机程序中，该程序可以自动从照片中设计浮雕珠宝。

Humans find seeing things effortless and this hides the fact that making sense of the visual world is a very difficult problem. Vision is difficult because each image we see could have been made by an infinite number of object and lighting combinations. Think of the simplest image property - grey level. The grey level of each pixel in an image is determined by the amount of light falling onto each object and the amount of light that is reflected back from each object. Dark objects have lower grey levels than light ones but even light objects have low grey levels when in shadow. We cannot tell whether we are looking at a dark object in bright light or a light object in shadow just by measuring grey levels. Even worse, when grey levels are different in different parts of an image we cannot tell if this difference is due to there being two objects or a change in the amount of light. Despite this problem humans are very good at working out the reasons for grey level changes; we CAN tell objects from shadows.One reason why we are so good at working out what's going on in images maybe that we use other properties such as colour and pattern to tell us what the grey levels mean. This idea has led to the concept of 'intrinsic images'. An intrinsic image is an image that describes one property of the scene. So instead of having a single image that mixes up shadows and object reflectances we might produce two intrinsic images one each for shadows and reflectance. Scientists have already succeeded in producing intrinsic images like these by using colour changes to work out what the grey levels mean. But, there is more than one type of shadow and more than one type of reflection. We want to improve on the existing methods by producing four intrinsic images instead of two. Our first intrinsic image will contain the type of gentle shading that is found on undulating surfaces. Our second intrinsic image will contain the hard shadows that are produced when an object blocks the light. Our third intrinsic image will describe the reflectance of matte objects and our forth image the reflections from shiny objects. To separate out these four images we will need to use additional information beyond colour. We think that surface patterns (e.g. wood grain) will provide the necessary information.Extracting four intrinsic images will be very helpful to those engineers who try to make computers understand what's going on in an image. To take just one example, humans seem to be very good at is estimating the shape of undulations on a surface from the way that it is shaded. We are so good at this that we do it automatically and the people who write computer software can trick us into thinking that their 'buttons' stand out from the screen just by adding a some highlights and shading to the edges. There are many computer programs that try to interpret shape-from-shading. While many of these programs work well they tend to assume that all changes in grey level are due to shading which is in tern due to surface undulations. We know that this assumption is not true in real pictures and these programs tend to do badly when looking at such images. But if we can produce shading only images from real images then these programs may work better.To test our ideas and decide on the best way achieve our desired results we will collect a large number of photographs of objects whose shape we either already know or can work out. We will calibrate these pictures very carefully and then use them to workout what information is conveyed by colour and pattern that can help us to workout the meaning of each grey level change. We will also test humans to see which cues they might be using. We will make our images available on the Internet so that others can try out their ideas too. We intend to work with a software company who will take the best of our ideas and implement them in a computer program that can automatically design embossed jewellery from photographs.