Physics-based colour models in computer vision, colour science, graphics and multimedia

点击翻译按钮获取中文摘要

This work is aimed at physics-based concepts in imaging. The project components described are specific, but form aspects of long-standing and deep problems throughout computer vision. The context for this research is the "fusion" of images. For example, suppose we need a greyscale ("black and white") xerox copy of a colour image: here we are aiming to fuse the three RGB (red, geen, blue) colour channels into a single grey output. This is a deceptively difficult problem, and hundreds of academic papers have been published on the subject. Many other problem domains are similar: e.g., fusing RGB plus Near-Infrared (NIR) into a new colour image with better night-vision properties for driving in the dark. It is understood in the camera-research community that including a NIR sensor in a leading brand of smartphones is about to take place. It turns out that the idea of "contrast" is critical for image fusion -- we need to capture local changes in the set of images to best combine them. Just what contrast consists of has been long-debated. In my Spectral Edge (SpE) application (and spin-off company) a definition of contrast is taken from solid mathematical foundations. It turns out the the SpE algorithm is applicable in a variety of situations: for example in medical imaging it is common to develop "image" data with more than just 3 dimensions like RGB colour. And in satellite imaging it is common to take "spectral" images with hundreds of values at each pixel location. The SpE approach disentangles how colour, on the one hand, and change in spatial position on the other, combine to form contrast. These calculations all take place in the "gradient" domain -- gradient is the derivative of the image, in both x and y directions, where derivative in imaging simply means the change from a pixel to its neighbouring pixel. Now, it is a classical problem in applied mathematics to transform back to a colour image the gradient (in each of R, G, B) that we arrived at by combining contrast. The classic solution for "reintegrating" the gradient back into image dates to the year 1800. However that method, even though since elaborated in a number of mathematical approaches, produces images that display unpleasant artifacts in the output image. Instead, in this proposal I will look deeper into making use of the SpE approach, along with a new, computer-science approach to reintegration into an output image. To date, the SpE method (and patent) uses mathematics that applies to a whole image: here I propose pixel-oriented methods that should produce substantially more detail. The methods developed will have far-reaching impact in that gradient-based approaches are now the standard approach to problems in computational photography. As one example, this research will have a major impact on the problem of bringing daytime image information into nighttime surveillance images.