Physics-based colour models in computer vision, colour science, graphics and multimedia

这项工作针对的是成像中基于物理的概念。所描述的项目*组件是特定的，但构成了整个计算机视觉中长期存在的和*深层次问题的各个方面。这项研究的背景是图像的“融合”。例如，假设我们需要一张彩色图像的灰度*(“黑白”)复印件：这里我们的目标是*将三个RGB(红、绿、蓝)颜色通道融合到单个灰色输出中。这是一个看似困难的问题，已经发表了数百篇关于这个主题的学术论文。许多其他问题领域也是类似的：例如，*将RGB和近红外(NIR)融合成一种新的彩色图像，具有更好的夜视*性能，适合在黑暗中驾驶。相机研究界了解到，在一款领先品牌的智能手机中安装近红外传感器即将*实现。事实证明，“对比度”的概念对于图像融合至关重要--我们需要捕捉图像集合中的*局部变化，以最佳地结合它们。到底*构成了什么反差，一直备受争议。在我的Spectral Edge(SPE)应用程序(以及*衍生公司)中，对比度的定义来自坚实的数学*基础。事实证明，SPE算法适用于各种*情况：例如，在医学成像中，开发具有超过3维的“图像”数据*的情况很常见，例如RGB颜色。在卫星成像中，拍摄每个像素位置都有数百个值的“光谱”图像是很常见的。*SPE方法一方面解开了颜色和空间位置的变化是如何结合在一起形成对比度的。*这些计算都发生在“梯度”域--梯度是图像在x和y方向上的*导数，在成像中导数*简单地表示从一个像素到其相邻像素的变化。现在，将我们通过组合对比度得到的梯度(在*R、G、B中)转换回彩色图像是应用数学中的一个经典问题。将渐变重新整合到图像中的经典解决方案可以追溯到1800年。然而，*这种方法，尽管后来在许多数学方法中进行了阐述，*产生的图像在输出图像中显示了令人不快的伪像*。相反，在这份提案中，我将更深入地研究如何使用SPE方法，以及一种新的计算机科学方法来重新整合到输出图像中。到目前为止，SPE方法(和专利)*使用了适用于整个图像的数学方法：这里我建议使用面向像素的*方法，这些方法应该产生更多的细节。*开发的方法将产生深远的影响，因为基于梯度的方法*现在是解决计算**摄影问题的标准方法。例如，这项研究将对将日间图像信息引入夜间监视*图像的*问题产生重大影响。

This work is aimed at physics-based concepts in imaging. The project***components described are specific, but form aspects of long-standing and***deep problems throughout computer vision. The context for this research***is the "fusion" of images. For example, suppose we need a greyscale***("black and white") xerox copy of a colour image: here we are aiming to***fuse the three RGB (red, geen, blue) colour channels into a single grey output. This is a***deceptively difficult problem, and hundreds of academic papers have been***published on the subject. Many other problem domains are similar: e.g.,***fusing RGB plus Near-Infrared (NIR) into a new colour image with better night-vision***properties for driving in the dark. It is understood in the camera-research***community that including a NIR sensor in a leading brand of smartphones is about***to take place.******It turns out that the idea of "contrast" is critical for image fusion -- we need to capture***local changes in the set of images to best combine them. Just what contrast***consists of has been long-debated. In my Spectral Edge (SpE) application (and***spin-off company) a definition of contrast is taken from solid mathematical***foundations. It turns out the the SpE algorithm is applicable in a variety of***situations: for example in medical imaging it is common to develop "image" data***with more than just 3 dimensions like RGB colour. And in satellite imaging it is***common to take "spectral" images with hundreds of values at each pixel location.***The SpE approach disentangles how colour, on the one hand, and change in spatial***position on the other, combine to form contrast.******These calculations all take place in the "gradient" domain -- gradient is the***derivative of the image, in both x and y directions, where derivative in imaging***simply means the change from a pixel to its neighbouring pixel. Now, it is a***classical problem in applied mathematics to transform back to a colour image ***the gradient (in each of***R, G, B) that we arrived at by combining contrast. The classic solution for***"reintegrating" the gradient back into image dates to the year 1800. However***that method, even though since elaborated in a number of mathematical approaches,***produces images that display unpleasant artifacts ***in the output image. Instead, in this proposal I will look deeper into making use***of the SpE approach, along with a new, computer-science approach to***reintegration into an output image. To date, the SpE method (and patent) ***uses mathematics that applies to a whole image: here I propose pixel-oriented***methods that should produce substantially more detail.******The methods developed will have far-reaching impact in that gradient-based***approaches are now the standard approach to problems in computational***photography. As one example, this research will have a major impact on the***problem of bringing daytime image information into nighttime surveillance***images.