Future Colour Imaging

彩色成像是日常生活的一部分。无论我们看电视，在平板电脑或手机上浏览内容，还是在工作中使用应用程序和软件，我们在屏幕上看到的内容都是几十年来色彩和图像研究的结果。未来的挑战是更多地了解内容图片。例如，在自动驾驶中，我们希望建立一个平台，可以独立于大气条件看到道路，我们不想在雾中驾驶时发生撞车。众所周知，在有雾的情况下，记录近红外信号的图像(与RGB相比)要清晰得多。什么是近红外？可见光谱有一个自然的彩虹顺序：紫罗兰色、深蓝色、蓝色、绿色、黄色、橙色和红色。红外线是继红色之后我们还看不到的“下一种颜色”。图像融合可以用来将RGB+NIR信号映射到我们可以看到的融合后的RGB对应信号。通过图像融合，相同的细节将在有雾或无雾的条件下呈现。有利的是，图像融合是一种工具，它将允许在现有的基于RGB的人工智能场景解释系统中合并和部署不可见信息，只需最少的再培训。我们的项目开始于光谱边缘图像融合方法，这是当前的领先技术。该方法和大多数图像融合算法通过合并4幅图像(RGB+NIR)的边缘来生成融合的RGB-Only 3通道边缘贴图。然后，边缘被变换(技术术语是重新整合)，以形成彩色图像。不幸的是，也是必然的，重新整合的图像经常具有诸如明亮的光晕、圆边或涂抹等缺陷。我们认为，这些缺陷是“边”如何定义的直接后果。在我们的研究中，我们将基于一种令人惊讶的数学洞察力来开发边缘的新定义，这是一件经过50年图像处理研究后相当大胆的事情！通过构造，重新整合的新边缘将有更少的光晕和涂抹的人工制品。然后，我们将使用我们改进的边缘表示和改进的图像融合算法来获得更好的图像。这些可能是融合后的图像本身：如果有智能双筒望远镜，当下雨或因距离而模糊的风景时，我们可以看到图像中的更多细节，不是很棒吗？然而，我们也相信，总的来说，摄影的未来是基于内容的，图像融合将帮助我们确定图像中的内容。例如，当我们在日落时拍摄一张照片时，场景中的阴影非常蓝。但是，在阴影之外，光线非常温暖(橙色)。这些场景的最佳图像再现涉及手动和差异地处理阴影和非阴影区域。在这里，我们试图自动找到图像中的光照内容。然后，在第二步中，我们将开发一个新的基于内容的框架来处理图像，因此，对于这个日落示例，我们不需要自己编辑照片。在互补性工作中，我们也有兴趣帮助人们看得更好。事实上，有很多研究表明，彩色滤镜可以帮助缓解视觉压力。彩色滤镜用于阅读障碍(有时会显著提高阅读速度)，现在有了蓝色吸收玻璃，它可以减少平板电脑显示器发出的蓝光(因为夜间的蓝光往往会让你睡不着)。这一领域的许多现有技术都是“直接的”。我们发现一种滤镜可以直接影响我们的观看方式(简单地说，如果我们在眼睛前面放一个黄色的滤镜，那么所有东西看起来都更黄)。我们的想法是设计与我们需要解决的任务相关的过滤器。对于配色的问题，我们将设计滤镜，以便如果您患有色盲，您将能够进行颜色匹配，就像您拥有正常的色觉一样。我们还将为蓝光问题和视觉压力开发间接的解决方案。

Colour Imaging is part of every day life. Whether we watch TV, browse content on our tablets or phones or use apps and software in our work the content we see on our screens is the result of decades of colour & imaging research. In the future, the challenge is to understand more about the content images. As an example, in autonomous driving we wish to build a platform that sees the road independent of the atmospheric conditions, we don't want to crash when we are driving in fog. It is well known that an image that records the near-infrared signal is much sharper (compared to RGB) in foggy conditions. What is near infrared? The visible spectrum has a natural rainbow order: Violet, Indigo, Blue, Green, Yellow Orange and Red. Infrared is the 'next colour' after red that we can't quite see. Image fusion can be used to map the RGB+NIR signal to a fused RGB counterpart, that we can see. Through image fusion the same detail will be present in foggy or non-foggy conditions. Advantageously, Image Fusion is a tool that will allow non visible information to be incorporated and deployed in existing RGB-based AI scene interpretation systems with minimal retraining. Our project begins with the Spectral Edge Image fusion method, the current leading technique. This method - and most image fusion algorithms - works by combining edges from the 4 images (RGB+NIR) to make a fused RGB-only 3-channel edge map. The edges are then transformed (the technical term is reintegrated) back to form a colour image. Unfortunately, and necessarily, the reintegrated images often have defects such as bright halos round edges or smearing. We argue that the defects are a direct consequence of how 'edges' are defined. In our research we will - based on a surprising mathematical insight - develop a new definition of edge, quite a bold thing to do after 50 years of image processing research! By construction the reintegrated new edges will have much less halo and smearing artefacts. We will then use our improved edge representation and improved image fusion algorithm to make better looking images. These might be the fused images themselves: wouldn't it be great to have smart binoculars that allow us to see more detail in images when it is rainy or a landscape that is blurred by distance. However, we also believe the future of photography, in general, is content-based and that image fusion will help us determine the content in an image. As an example, when we take a picture at sunset, the shadows in the scene are very blue. But, outside of the shadow the light is very warm (orangish). The best image reproductions for these scenes involves manually and differentially processing shadow and non shadow regions. Here, we seek to find the illumination content in image automatically. Then in a second step we will develop a new content-based framework for manipulating images so that, for this sunset example, we don't need to edit the photos ourselves. In complementary work, we are also interested in helping people see better. Indeed, there is a lot of research that demonstrates that coloured filters can help mitigate visual stress. Coloured filters are used in Dyslexia (sometimes leading to dramatic improvements in reading speed) and there is now blue absorbing glass which will reduces the blue light coming from a tablet display (since blue light at night tends to keep you awake). Much of the prior art in this area is 'direct'. We find a filter to directly impact on how we see (simply, if we put a yellow filter in front of the eye then everything looks more yellow). Our idea is to deign filters that are related to the tasks we need to solve. For the problem of matching colours we will design filters so that if you suffer from colour-blindness you will be able to colour match as if you had normal colour vision. We will also develop indirect solutions for the 'blue light' problem and visual stress.