Colour space homography

对于商业“成像”产品，如手机中的数码相机，工程师们寻求简单而优雅的解决方案来解决难题。你会注意到，新的相机比旧的相机能拍出更好的照片。这在一定程度上是由于硬件的不断进步：处理器更快，像素更多。然而，已经有了一些重要的解决问题的见解，使解决迄今难以解决的问题成为可能。这方面的一个很好的例子是颜色投射问题(该提案的焦点之一)。当你用相机拍照时，所记录的颜色取决于场景中物体的颜色，以及(可能令人惊讶的)光线的颜色。从身体上看，你的白色T恤在阳光直射下或在阴影中时是淡黄色和蓝色的(因为阳光和阴影分别是黄色和蓝色的)。我们看不到色彩投射，因为我们的视觉系统将光线的颜色分解出来，同样，数码相机(在大多数情况下)通过处理图像来去除色彩投射。但是，如果你将今天的相机输出与15年前的相机进行比较，你会发现现代相机在消除色彩投射方面要好得多。为什么？由于聪明的洞察力帮助工程师建立了正确推断主流光线颜色的系统。这项研究项目以一种令人惊讶的新观察开始，我们相信这将帮助我们进一步改进成像产品，如彩色相机。我们第一次在图像中记录的颜色--通常是三个R、G和B数字，用来衡量像素的红、绿和蓝--与世界上点的3D位置以及这些点与图像中像素位置的对应关系之间建立了深刻的联系。考虑到几何学，我们都意识到火车轨道似乎在远处汇合，尽管我们知道它们保持相同的距离(本质上，我们理解3D世界是如何映射到2D图像的)。在这项研究中，我们首先展示了颜色之间的关系-在不同的观看条件下观看相同颜色的对象-与几何学中连接不同视点的关系完全相同--比如建筑物的正面。在数学上，这种关系被称为同形异形。重要的是，现在我们已经将你在相机中看到的颜色与3D世界如何映射到图像联系在一起，我们可以利用这种观察--我们可以利用在几何领域中学到的教训--来帮助我们解决一些彩色成像问题。首先，通过利用色彩空间单应性，相机制造商(和爱好者)将能够更准确地校准他们的相机，这将进一步优化图像的色彩保真度。单应对颜色偏色去除问题也起着重要的作用，在这个提案中，我们将开发一种基于单应的算法，这将推动最新技术的发展。除了摄影，我们还对基于颜色的医疗诊断等问题感兴趣，例如，我们将致力于改进皮肤病变的自动识别。我们提出，单应也是解决高级视觉任务的关键，包括识别、处理和消除图像中的阴影。更新奇的是，我们发现，回答“我们如何找到彩色滤光器，当放置在相机前面时，使相机以类似于人类视觉系统的方式测量光线”的问题也涉及到单应问题的求解。单应图与图像融合等问题有关--将数百幅图像融合成一个颜色摘要--在所谓的派生域中处理图像。在这里，图像被转换为边缘表示，然后在重新整合输出之前对这些边缘进行处理。同形图有望使重新融入过程更快，更不容易引入人工制品(这是现有技术的一个众所周知的问题)。

For commercial 'imaging' products like the digital camera in your phone, engineers seek simple and elegant solutions to hard problems. You will have noticed that newer cameras take better pictures than older ones. This is in part due to ever advancing hardware: there are faster processors and more pixels. Yet, there has been some important problem solving insights that make it possible to solve problems that were hitherto intractable. A good example of this is the problem of colour casts (one of the foci of this proposal). When you take a picture with a camera the colours that are recorded are dependent on the colour of objects in the scene and (perhaps surprisingly) on the colour of the light. Physically, your white T-shirt is yellowish and bluish when viewed in direct sunlight or when you are in the shadows (because sunlight and shadows are respectively yellowish and bluish). We do not see the colour casts as our vision system factors out the colour of the light and, likewise, digital cameras process images to (much of the time) remove colour casts. But, if you compare the outputs of cameras today with those 15 years ago, the modern era cameras are much better at removing colour casts. Why? because of clever insights that helped engineers build systems which correctly infer the colour of the prevailing light.This research project begins with a surprising new observation which we believe will help us improve still further imaging products such as colour cameras. We make, for the first time, a deep link between the colours recorded in image - typically three R,G and B numbers that measure the redness, greenness and blueness of a pixel - and the 3D locations of points in the world and how these points correspond to pixel locations in an image. Thinking about geometry, we are all aware that train tracks appear to converge in the distance even although we know they stay the same distance apart (intrinsically, we understand how the 3D world maps to 2D pictures). In this research we begin by showing that the relationship between the colours - same colourful object viewed under different viewing conditions - is exactly the same as the relationship that links different viewpoints - say of the front of a building - in geometry. Mathematically, this relationship is called an homography.Importantly, now that we have linked the colours you see in the camera to how the 3D world maps to images we can use this observation - we can exploit lessons learnt in the geometric domain - to help us solve some colour imaging problems. First, by exploiting colour space homography, camera manufacturers (and enthusiasts) will be able to more accurately calibrate their cameras and this will further optimize image colour fidelity. Homographies also play an important role the colour cast removal problem and in this proposal we will develop a homography-based algorithm that will advance the state of the art. Outside of photography, we are interested in problems such as colour based medical diagnosis e.g. we will be aiming to improve the automatic identification of skin lesions. We propose that homographies are also the key to solving high level vision tasks including identifying, manipulating and removing shadows from images.More novelly, we have found that answering the question "how do we find a coloured filter which, when place in front of a camera makes the camera measure light in a way that is similar to the human visual system' also involves solving for a homography. Homographies have a link to problems such as image fusion - fusing 100s of images into a single colour summary - which process images in the so-called derivative domain. Here images are transformed into an edge representation and then these edges are manipulated before an output is reintegrated. Homographies hold the promise of making the reintegration process faster and less prone to introducing artifacts (a well known problem of existing techniques).