Real-Tune 3D Shape Reconstruction, Visualization, Editing, and Coding for 3D Video

实现三维视频生成、可视化、编辑、编码的技术成果如下：PC集群系统（PC机：30台，活动摄像头：25台），可同步捕捉人体动作的多视角视频。准定视点平移变焦相机的精确标定方法基于PC集群的实时动态三维形状重建的3基面视锥相交算法及其并行流水线处理方法使用该系统，我们可以在2cm X 2cm X 2cm的空间分辨率下每秒重建超过10个人体活动。人体部位放大主动视频采集的协同摄像机工作规划使用这种方法，我们将能够显著提高三维形状和表面纹理的精度。三维变形网格模型用于重建精确的三维人体形状和运动；利用这种方法，我们可以重建三维人体的均匀凹部。基于视点的三维视频可视化高保真纹理映射算法使用这种方法，我们可以可视化几乎标准视频质量的3D视频。使用多个朗伯球的照明环境感知（即多个光源的估计）8。使用骨架立方体的照明环境传感。采用三维视频和全方位全景视频的交互式三维视频编辑系统基于展开正多面体的全向视频编码方法。基于三维曲面形状平面映射的三维视频编码方法对于上述两种编码方法，我们将其提交给MPEG进行国际标准化。

We have obtained the following technical attainments to realize 3D video generation, visualization, editing, and coding.1. PC cluster system (PC : 30, active camera : 25) to capture synchronized multi-view video of human action2. Precise calibration method for quasi fixed-viewpoint pan-tilt-zoom camera3. 3 base plane visual cone intersection algorithm and its parallel pipeline processing method for real-time dynamic 3D shape reconstruction using PC cluster ; with this system, we can reconstruct over 10 volumes per second of human action at 2cm X 2cm X 2cm spatial resolution.4. cooperative camera work planning for zoom-up active video capturing of human body parts ; with this method, we will be able to increase the accuracy of 3D shape and surface texture significantly.5. 3D deformable mesh model for reconstructing accurate 3D human shape and motion ; with this method, we can reconstruct even concave portions of 3D human body.6. Viewpoint dependent high fidelity texture mapping algorithm for 3D video visualization ; with,this method, we can visualize 3D video of almost standard video quality.7. Lighting environment sensing (i.e. estimation of multiple light sources) using multiple Lambertian spheres8. Lighting environment sensing using skeleton cube9. Interactive 3D video editing system using 3D video and omni-directional panoramic video10. Coding method of omni-directional video based on unfolded regular polyhedron11. Coding method of 3D video based on planar mapping of 3D surface shapeAs for two coding methods listed above, we proposed them to MPEG for international standardization.