沉浸式3D全景视频的表示编码及传输

近年来，各个国家和跨国企业不断在虚拟现实产业进行重点布局。而3D全景视频作为目前虚拟现实视频的主要内容源，其拍摄架构、拼接质量、编码性能和传输效率等环节直接影响着虚拟现实产业的发展。本项目着眼于研究和设计沉浸式3D全景视频通信的框架和系统。首先，课题研究并设计相应的3D全景相机标定系统，同时兼顾了标定的速度和准确性；为了实现较好的无缝式拼接和交互式体验，课题提出了相应的颜色平衡调节算法和基于眼球跟踪的三层自主式变焦体系。在模型表示方面，课题研究了几种常用的模型，并设计了模型转换过程中的上下采样架构，以提高转换性能。在视频编码方面，设计了非对称编码框架、视角自适应编码算法和混合仿射变换的运动估计算法。最后，综合带宽、延迟、视角切换概率和缓存区大小设计切片策略和传输架构，最终实现一套完整的沉浸式3D全景视频的表示编码及传输体系，为虚拟现实产业提供稳定的内容和通信架构。

Recently, many countries and international corporations put in large energy and investment on virtual reality (VR) industry. 3D panorama video, as the main source contents of VR, its capturing framework, calibration and stitching performance, compression efficiency, as well as transmission system, has direct effect on VR industry. Our project focuses on the research of framework designing and system structure of 3D panorama video communication. First, we design a calibration system and algorithm for 3D panorama video cameras, considering both the accuracy and complexity. In order to provide seamless and interactive performance, we propose color balance tuning algorithm and three-layer focus adapting framework based on gaze tracking. Second, by researching the popular panorama model, we propose corresponding model converting algorithms considering down/up-sampling process, as well as the compatibility with subsequent compression algorithm. Third, we propose asymmetric encoding, view-adaption encoding and affine-hybrid motion estimation into 3D panorama video compression. Finally, considering the bandwidth, delay time, probability of view switching and buffer size, a segmentation strategy is proposed for video transmission. An integral immersive 3D panorama video representation, compression and transmission system are formed so as to provide better source content and transmission framework for virtual reality industry.