Physics-based colour models in computer vision, colour science, graphics and multimedia

这项工作的目的是在成像的物理为基础的概念。项目 所描述的组件是具体的，但形成长期存在的方面， 计算机视觉中的深层问题。 本研究的背景 是图像的“融合”。 例如，假设我们需要一个灰度 （“黑与白色”）彩色图像的复印：这里我们的目标是 将三个RGB（红、绿、蓝）颜色通道融合成单个灰色输出。 这是一 这是一个看似困难的问题，数百篇学术论文已经被 发表在这个主题上。 许多其他问题域也是类似的：例如， 融合RGB和近红外（NIR）到一个新的彩色图像，具有更好的夜视能力 在黑暗中驾驶的性能。据了解，在相机研究 社区，包括近红外传感器在智能手机的领先品牌是关于 发生。 事实证明，“对比度”的概念对于图像融合至关重要--我们需要捕捉到 图像集中的局部变化以最佳地联合收割机组合它们。 什么样的对比 这是一个长期争论的问题。 在我的光谱边缘（SpE）应用程序中（以及 分拆公司）对比度的定义取自坚实的数学 层. 事实证明，SPE算法适用于各种 情况：例如，在医学成像中， 像RGB颜色那样的三维空间。在卫星图像中， 通常在每个像素位置拍摄具有数百个值的“光谱”图像。 SPE方法一方面解开了颜色如何在空间中变化， 另一方面，联合收割机形成对比。 这些计算都发生在“梯度”域中--梯度是 图像在x和y方向上的导数，其中成像中的导数 简单地表示从像素到其相邻像素的变化。 现在，它是一个 应用数学中的一个经典问题，将图像转换回彩色图像 梯度（在每个 R，G，B），这是我们通过结合对比度得到的。经典的解决方案 将梯度“重新整合”到图像中可以追溯到1800年。 然而 这种方法，尽管在许多数学方法中得到了阐述， 产生显示令人不快的伪影的图像 在输出图像中。 相反，在本提案中，我将深入研究如何利用 沿着一种新的计算机科学方法， 重新整合到输出图像中。 迄今为止，SpE方法（和专利） 使用适用于整个图像的数学：在这里，我建议面向像素 这些方法应该产生更多的细节。 所开发的方法将对基于梯度的 方法现在是计算问题的标准方法， 摄影.作为一个例子，这项研究将对 将白天图像信息纳入夜间监控的问题 图像.

This work is aimed at physics-based concepts in imaging. The project components described are specific, but form aspects of long-standing and deep problems throughout computer vision. The context for this research is the "fusion" of images. For example, suppose we need a greyscale ("black and white") xerox copy of a colour image: here we are aiming to fuse the three RGB (red, geen, blue) colour channels into a single grey output. This is a deceptively difficult problem, and hundreds of academic papers have been published on the subject. Many other problem domains are similar: e.g., fusing RGB plus Near-Infrared (NIR) into a new colour image with better night-vision properties for driving in the dark. It is understood in the camera-research community that including a NIR sensor in a leading brand of smartphones is about to take place. It turns out that the idea of "contrast" is critical for image fusion -- we need to capture local changes in the set of images to best combine them. Just what contrast consists of has been long-debated. In my Spectral Edge (SpE) application (and spin-off company) a definition of contrast is taken from solid mathematical foundations. It turns out the the SpE algorithm is applicable in a variety of situations: for example in medical imaging it is common to develop "image" data with more than just 3 dimensions like RGB colour. And in satellite imaging it is common to take "spectral" images with hundreds of values at each pixel location. The SpE approach disentangles how colour, on the one hand, and change in spatial position on the other, combine to form contrast. These calculations all take place in the "gradient" domain -- gradient is the derivative of the image, in both x and y directions, where derivative in imaging simply means the change from a pixel to its neighbouring pixel. Now, it is a classical problem in applied mathematics to transform back to a colour image the gradient (in each of R, G, B) that we arrived at by combining contrast. The classic solution for "reintegrating" the gradient back into image dates to the year 1800. However that method, even though since elaborated in a number of mathematical approaches, produces images that display unpleasant artifacts in the output image. Instead, in this proposal I will look deeper into making use of the SpE approach, along with a new, computer-science approach to reintegration into an output image. To date, the SpE method (and patent) uses mathematics that applies to a whole image: here I propose pixel-oriented methods that should produce substantially more detail. The methods developed will have far-reaching impact in that gradient-based approaches are now the standard approach to problems in computational photography. As one example, this research will have a major impact on the problem of bringing daytime image information into nighttime surveillance images.