Modelling Image Formation and In-Camera Imaging Pipelines

点击翻译按钮获取中文摘要

My research program is focused on understanding the physical world through images captured by digital cameras. More specifically, I'm interested in what an image can tell us about the real-world environment. To this end, my research focuses on ways to model how physical light coming into the camera is transformed into the final three-channel image (red, green, blue) pixel-based images. This type of image formation is sometimes referred to as "low-level computer vision", which deals with how images are formed and their relationship to the physical world. Low-level computer vision often treats an image as a 2D signal, much like a communication signal. In this way, we can discuss problems such as trying to determine the true signal when it has undergone some degradation (such as sensor noise, or movement during the image process – e.g. image blur due to camera motion). Such image formation models can also take into consideration environment factors, such as fog and haze, and how this can be removed through computational methods.******As part of this research program, one of the key components of my work is understanding exactly how digital cameras work. While we like to think of cameras as light-measuring devices, the current design of commodity cameras includes a great deal of additional processing that is applied on board the camera (often referred to as the in-camera processing pipeline). The goal of most camera manufacturers is to make visually pleasing photographs and not necessarily to faithfully capture the imaged scene. While this is ideal for photography, this type of image manipulation is often at odds with models used in low-level computer vision. In particular, in-camera manipulation can modify colours, change local contrast, and substantially distort the original sensor response in a way that makes it challenging to determine the actual nature of the physical environment. One of my research focuses is to design new camera processing pipelines that allow the ability to produce both photographic images and images suitable for scientific purposes. Developing such "hybrid cameras" has the potential for significant impact, as we now are using our cameras (especially those on our mobile devices) for many non-photo-centric tasks (such as document scanning, object identification, medical imaging, colour matching). The long-term prospects of this research program are to shape the future design of consumer cameras and the applications they can be used for.