Multiview Video Coding, Processing and Applications

点击翻译按钮获取中文摘要

3D imaging and broadcasting technologies have drawn a great deal of attention from the industry and the research community, and represent an important future growth area of digital media industry and services. These technologies aim to offer consumers much enhanced and immersive experience in watching videos, and therefore could lead to a multitude of new multimedia applications for education, medicine, surveillance and entertainment. One way to achieve 3D imaging is to use multi-view video (MVV), where the system uses several cameras to acquire multiple video sequences of the same scene simultaneously. The data are then compressed and transmitted to the users. In this project, we will study and develop algorithms for multi-view video coding, 3D modeling, multi-view video processing and analysis, and multi-view based resolution enhancement. In particular we will devise compression algorithm(s) capable of removing all three types of redundancies in MVV sequences: intraframe, interframe and inter-view. Also, for the transmission of MVV via heterogeneous networks our MVV codec will be made scalable. Existing MVV systems require dedicated camera arrays and the corresponding expensive controlling hardware and software, making the mass deployment of these systems difficult and impractical, especially in the consumer markets. In this project, we propose a low-cost user-driven client-server alternative to these high-end 3D systems that can provide some basic multi-view video functionalities. The spatial, temporal and perspective redundancies in multi-view video suggest a great potential of improving video quality by superresolution techniques. Through motion estimation and geometric warping we can register multiple views in subpixel precision and upsample each of these views to a higher spatial resolution. We will formulate and investigate a new inverse problem for multi-view superresolution reconstruction, and develop an algorithm to solve the problem. This project will result in the development of new scientific knowledge and perhaps intellectual property that could ultimately be commercialized. Another important aspect of the project will be the development of highly qualified personnel.