Estimating the intrinsic characteristics of real images to aid analysis

人类可以毫不费力地看到事物，这掩盖了这样一个事实，即理解视觉世界是一个非常困难的问题。视觉是困难的，因为我们看到的每一张图像都可能是由无限数量的物体和灯光组合构成的。想想最简单的图像属性--灰度级。图像中每个像素的灰度级由投射到每个对象上的光量和从每个对象反射回来的光量决定。暗物体的灰度级比浅色物体低，但即使是浅色物体，在阴影中也有低灰度级。我们不能仅仅通过测量灰度级来判断我们在明亮的光线下看的是暗物体还是在阴影中看的光物体。更糟糕的是，当图像不同部分的灰度级不同时，我们无法判断这种差异是由于存在两个对象还是由于光量的变化。尽管有这个问题，人类还是很擅长找出灰度级变化的原因；我们可以区分物体和阴影。我们之所以如此擅长弄清楚图像中发生的事情，其中一个原因可能是我们使用其他属性，如颜色和图案来告诉我们灰度级意味着什么。这一理念催生了“内在意象”的概念。固有图像是描述场景的一种属性的图像。因此，我们可以生成两个本征图像，每个图像的阴影和反射率各一个，而不是只有一个图像将阴影和物体反射率混合在一起。科学家们已经成功地利用颜色变化来计算出灰度级的含义，从而成功地产生了这样的内在图像。但是，存在不止一种类型的阴影和不止一种类型的反射。我们希望通过生成四个本征图像而不是两个来改进现有的方法。我们的第一个内在图像将包含在起伏表面上发现的温和阴影类型。我们的第二个本征图像将包含当物体遮挡光线时产生的硬阴影。我们的第三个固有图像将描述哑光对象的反射率，而我们的第四个图像将描述来自有光泽的对象的反射。为了区分这四幅图像，我们需要使用颜色以外的其他信息。我们认为表面图案(例如木纹)将提供必要的信息。提取四个固有图像将对那些试图使计算机理解图像中发生的事情的工程师非常有帮助。仅举一个例子，人类似乎非常擅长从阴影的方式来估计表面上的波纹形状。我们在这方面做得如此之好，以至于我们会自动完成这项工作，编写计算机软件的人可以欺骗我们，只需在屏幕上添加一些高亮和阴影，就可以让我们认为他们的按钮从屏幕上脱颖而出。有许多计算机程序试图解释从明暗到明暗的形状。虽然这些程序中的许多都运行良好，但它们倾向于认为所有灰度级的变化都是由于表面起伏造成的阴影。我们知道，这一假设在真实图片中是不正确的，这些程序在观看此类图像时往往表现不佳。但如果我们能从真实图像中生成仅有阴影的图像，那么这些程序可能会运行得更好。为了测试我们的想法并决定以最佳方式实现我们想要的结果，我们将收集大量物体的照片，这些物体的形状我们已经知道或可以计算出来。我们将非常仔细地校准这些图片，然后使用它们来计算出颜色和图案所传达的信息，从而帮助我们计算出每个灰度变化的含义。我们还将测试人类，看看他们可能会使用哪些线索。我们将把我们的图片放在互联网上，这样其他人也可以尝试他们的想法。我们打算与一家软件公司合作，该公司将吸收我们最好的想法，并将其在计算机程序中实施，该程序可以根据照片自动设计浮雕珠宝。

Humans find seeing things effortless and this hides the fact that making sense of the visual world is a very difficult problem. Vision is difficult because each image we see could have been made by an infinite number of object and lighting combinations. Think of the simplest image property - grey level. The grey level of each pixel in an image is determined by the amount of light falling onto each object and the amount of light that is reflected back from each object. Dark objects have lower grey levels than light ones but even light objects have low grey levels when in shadow. We cannot tell whether we are looking at a dark object in bright light or a light object in shadow just by measuring grey levels. Even worse, when grey levels are different in different parts of an image we cannot tell if this difference is due to there being two objects or a change in the amount of light. Despite this problem humans are very good at working out the reasons for grey level changes; we CAN tell objects from shadows.One reason why we are so good at working out what's going on in images maybe that we use other properties such as colour and pattern to tell us what the grey levels mean. This idea has led to the concept of 'intrinsic images'. An intrinsic image is an image that describes one property of the scene. So instead of having a single image that mixes up shadows and object reflectances we might produce two intrinsic images one each for shadows and reflectance. Scientists have already succeeded in producing intrinsic images like these by using colour changes to work out what the grey levels mean. But, there is more than one type of shadow and more than one type of reflection. We want to improve on the existing methods by producing four intrinsic images instead of two. Our first intrinsic image will contain the type of gentle shading that is found on undulating surfaces. Our second intrinsic image will contain the hard shadows that are produced when an object blocks the light. Our third intrinsic image will describe the reflectance of matte objects and our forth image the reflections from shiny objects. To separate out these four images we will need to use additional information beyond colour. We think that surface patterns (e.g. wood grain) will provide the necessary information.Extracting four intrinsic images will be very helpful to those engineers who try to make computers understand what's going on in an image. To take just one example, humans seem to be very good at is estimating the shape of undulations on a surface from the way that it is shaded. We are so good at this that we do it automatically and the people who write computer software can trick us into thinking that their 'buttons' stand out from the screen just by adding a some highlights and shading to the edges. There are many computer programs that try to interpret shape-from-shading. While many of these programs work well they tend to assume that all changes in grey level are due to shading which is in tern due to surface undulations. We know that this assumption is not true in real pictures and these programs tend to do badly when looking at such images. But if we can produce shading only images from real images then these programs may work better.To test our ideas and decide on the best way achieve our desired results we will collect a large number of photographs of objects whose shape we either already know or can work out. We will calibrate these pictures very carefully and then use them to workout what information is conveyed by colour and pattern that can help us to workout the meaning of each grey level change. We will also test humans to see which cues they might be using. We will make our images available on the Internet so that others can try out their ideas too. We intend to work with a software company who will take the best of our ideas and implement them in a computer program that can automatically design embossed jewellery from photographs.