Real-Tune 3D Shape Reconstruction, Visualization, Editing, and Coding for 3D Video

在实现3D视频的生成、可视化、编辑、编码等方面取得了以下技术成果：1. PC集群系统（PC：30，活动摄像头：25）捕捉同步的多视角视频的人的行动2.准固定视点云台变焦相机的精确标定方法。基于PC集群的实时动态三维形状重建的基平面视锥求交算法及其并行流水线处理方法，该系统可以在2cm × 2cm × 2cm的空间分辨率下，每秒重建10个以上的人体动作体.协同摄像机工作规划用于人体部位的放大主动视频捕获，通过该方法，我们将能够显著提高3D形状和表面纹理的准确性.三维可变形网格模型，用于重建精确的三维人体形状和运动;使用该方法，我们可以重建甚至凹部的三维人体.视点相关的高保真纹理映射算法用于3D视频可视化;使用该方法，我们可以可视化几乎标准视频质量的3D视频.使用多个朗伯球的照明环境感测（即多个光源的估计）8。使用骨架立方体的照明环境感测9.使用3D视频和全方位全景视频的交互式3D视频编辑系统10.基于展开正多面体的全方位视频编码方法。基于3D表面形状平面映射的3D视频编码方法对于上述两种编码方法，我们已将其提交给MPEG进行国际标准化。

We have obtained the following technical attainments to realize 3D video generation, visualization, editing, and coding.1. PC cluster system (PC : 30, active camera : 25) to capture synchronized multi-view video of human action2. Precise calibration method for quasi fixed-viewpoint pan-tilt-zoom camera3. 3 base plane visual cone intersection algorithm and its parallel pipeline processing method for real-time dynamic 3D shape reconstruction using PC cluster ; with this system, we can reconstruct over 10 volumes per second of human action at 2cm X 2cm X 2cm spatial resolution.4. cooperative camera work planning for zoom-up active video capturing of human body parts ; with this method, we will be able to increase the accuracy of 3D shape and surface texture significantly.5. 3D deformable mesh model for reconstructing accurate 3D human shape and motion ; with this method, we can reconstruct even concave portions of 3D human body.6. Viewpoint dependent high fidelity texture mapping algorithm for 3D video visualization ; with,this method, we can visualize 3D video of almost standard video quality.7. Lighting environment sensing (i.e. estimation of multiple light sources) using multiple Lambertian spheres8. Lighting environment sensing using skeleton cube9. Interactive 3D video editing system using 3D video and omni-directional panoramic video10. Coding method of omni-directional video based on unfolded regular polyhedron11. Coding method of 3D video based on planar mapping of 3D surface shapeAs for two coding methods listed above, we proposed them to MPEG for international standardization.