Embedded vision systems - a platform for integrating modern imaging sensors and real-time image processing

点击翻译按钮获取中文摘要

Megapixel image sensors are commonplace in our modern society and are found in devices such as mobile phones, game consoles, still and video cameras. The fierce competition between sensor manufacturers that operate in this large market segment has driven down costs and at the same time enabled and accelerated the development of sensors with novel capabilities. This revolution in sensor technology has made very large arrays with wavelength limited pixel size available in consumer products. At the same time problematic issues such as low sensitivity and high noise levels have been addressed successfully to the point that the sensors become of interest to the more demanding scientific and industrial community for a very diverse range of applications. The resulting high signal to noise ratio and high frame rate capability of emerging sensors enable important imaging tasks that require high dynamic range (HDR). A further exciting development is that of adding a further dimension to the 2D imaging by measuring the 'colour' of the light (spectroscopic) incident on each pixel. A nondestructive readout of the sensor during exposure allows very large gains in the dynamic range of a sensor albeit at the expense of having to read out the sensor multiple times for a single HDR image. The combination of 3D, high resolution, high dynamic range and high frame rate is exacerbating an already serious issue for handling and transmitting data at very modest frame rates of 30 per second. Many existing professional cameras completely fill the bandwidth that the highest speed data transfer protocols offer. Bandwidth issues aside, the task of image processing remains: simply filling large memory buffers with data for offline processing doesn't suit many applications.To address this serious issue of data avalanche in our EPSRC funded work on transparent imaging and diagnostics of highly energetic particle beams, we took inspiration from nature. In human vision, for example, the information from the light sensitive cells does not immediately go to the brain, pixel by pixel, but already in the retina a certain amount of data is digested by combining the signals from several visual receptors. A sophisticated front-end image processing approach was the only way to measure the key high-resolution beam characteristics at high frame rates. After three generations of camera designs and 10 man years of effort, we have refined a novel camera architecture that combines both coarse and fine grain parallel image processing using a heterogeneous hierarchy of image processing elements. With these it becomes standard to only transmit those pixels that vary frame by frame. We have already implemented various dedicated realtime image processing tasks such as reducing data flow by only passing on that part of the image that changes between frames in pixel intensity and/or colour, HDR imaging, particle tracking and size measurement, colour interpolation, noise reduction, feature extraction, histogramming, and many others. The work we propose in this project will allow us to show the market what our systems are capable of and to provide selected groups that show an interest with the systems. We will demonstrate how our approach will revolutionize many sophisticated imaging techniques in much the same way as digital still cameras have transformed photography. We will turn our devices to 'plug and play' camera systems that can readily be used by others. This project brings together specialists from both academic and commercial background to prepare our vision systems technology for market and to investigate the best route to commercial exploitation of the created IP and know-how. Together the partners will implement two key applications: spectroscopic imaging and integrate the device as the brain and eyes of an unmanned vehicle initiative at Manchester. On completion of the project we will be in a strong position to secure support from venture capital or seed funds.