Enabling technologies of the video systems of future

点击翻译按钮获取中文摘要

Recently, an unprecedented and heterogeneous range of new imaging devices such as image+depth sensors, light-field cameras and holographic devices have enabled new capture and visualization perspectives. This is driving a disruptive paradigm shift in the acquisition and consumption of visual content: moving from a planar, 2D representation towards imaging in volumetric and immersive modalities. To enable these novel representations, a mathematical framework called ``plenoptic" that considers the usual color information of any point within a scene and adds directional information about the colour change when observed from different positions has been developed. Many scientific challenges remain for researchers to address before plenoptic modalities such as light field, hologram and point cloud become prevalent. From an acquisition point of view, plenoptic imaging requires the ability to capture a large amount of high dimensional data. On the one hand, this higher dimensional representation of visual data offers powerful capabilities and has the potential to substantially improve the performance of traditional computer vision tasks. On the other hand, the high-dimensionality of plenoptic also brings up new challenges in terms of data capture, data compression, content editing, and display. This research program seeks to develop technologies for acquisition, compression, communication and display of different plenoptic modalities. In the short term we will explore scalable coding of holographic and light field data which will facilitate the extraction of different levels of quality, spatial resolution, depth resolution and number of viewing angles from the codestream. We will also investigate frame rate up conversion of light field video and the application of light field imaging for object tracking. The development of proper compression and processing technologies proposed in this research program will enable and increase the popularity of plenoptic modalities. The new knowledge arising from this research will find its way into new technologies and products in areas of great importance to Canada such as healthcare, education and auto industry.