Multiview Video Coding, Processing and Applications

3D成像和广播技术已经引起了业界和研究界的高度关注，代表着数字媒体产业和服务未来重要的增长领域。这些技术旨在为消费者提供更强的沉浸式观看视频的体验，因此可能会导致大量新的多媒体应用于教育、医疗、监控和娱乐。实现3D成像的一种方法是使用多视点视频（MVV），其中系统使用多个摄像机同时获取同一场景的多个视频序列。然后将数据压缩并传输给用户。在本项目中，我们将研究和开发多视点视频编码、3D建模、多视点视频处理和分析以及基于多视点的分辨率增强算法。特别是，我们将设计压缩算法(s)能够消除所有三种类型的冗余在MVV序列：帧内，帧间和视图间。此外，为了通过异构网络传输MVV，我们的MVV编解码器将具有可扩展性。现有的MVV系统需要专用的相机阵列和相应的昂贵的控制硬件和软件，使得这些系统的大规模部署变得困难和不切实际，特别是在消费市场。在这个项目中，我们提出了一个低成本的用户驱动的客户端-服务器替代这些高端3D系统，可以提供一些基本的多视图视频功能。多视点视频的空间、时间和视角冗余表明，超分辨率技术在提高视频质量方面具有巨大的潜力。通过运动估计和几何翘曲，我们可以在亚像素精度下注册多个视图，并将每个视图上采样到更高的空间分辨率。我们将制定和研究一个新的多视图超分辨率重建反问题，并开发一种算法来解决这个问题。这个项目将带来新的科学知识的发展，也许还有最终可以商业化的知识产权。该项目的另一个重要方面是培养高素质的人才。

3D imaging and broadcasting technologies have drawn a great deal of attention from the industry and the research community, and represent an important future growth area of digital media industry and services. These technologies aim to offer consumers much enhanced and immersive experience in watching videos, and therefore could lead to a multitude of new multimedia applications for education, medicine, surveillance and entertainment. One way to achieve 3D imaging is to use multi-view video (MVV), where the system uses several cameras to acquire multiple video sequences of the same scene simultaneously. The data are then compressed and transmitted to the users. In this project, we will study and develop algorithms for multi-view video coding, 3D modeling, multi-view video processing and analysis, and multi-view based resolution enhancement. In particular we will devise compression algorithm(s) capable of removing all three types of redundancies in MVV sequences: intraframe, interframe and inter-view. Also, for the transmission of MVV via heterogeneous networks our MVV codec will be made scalable. Existing MVV systems require dedicated camera arrays and the corresponding expensive controlling hardware and software, making the mass deployment of these systems difficult and impractical, especially in the consumer markets. In this project, we propose a low-cost user-driven client-server alternative to these high-end 3D systems that can provide some basic multi-view video functionalities. The spatial, temporal and perspective redundancies in multi-view video suggest a great potential of improving video quality by superresolution techniques. Through motion estimation and geometric warping we can register multiple views in subpixel precision and upsample each of these views to a higher spatial resolution. We will formulate and investigate a new inverse problem for multi-view superresolution reconstruction, and develop an algorithm to solve the problem. This project will result in the development of new scientific knowledge and perhaps intellectual property that could ultimately be commercialized. Another important aspect of the project will be the development of highly qualified personnel.