Construction of System of 3-D Modeling and Virtual Image Generation for Mixed Reality

本研究旨在为计算机视觉和混合现实提供新的可能性，包括以下几个方面的内容。其中包括基于神经网络的三维建模和虚拟图像生成、扫描电子显微镜阴影图像的形状提取、形状识别和图像库中自由视点图像的生成、基于稳健背景差分的快速光照变化运动目标提取方法以及基于粒子滤波的固定视点下运动跟踪方法的改进和高效实现。利用神经网络对彩色目标图像进行扩展，利用形状信息恢复颜色反射因子，生成任意光源方向、任意视点的虚拟图像，以适应混合现实。与以往的基于CG模型的方法相比，改进的基于神经网络的方法具有更好的真实性。以Hopfield神经网络为优化方法，建立了从扫描电子显微镜中恢复形状的方法。提出了一种从CG图像库中利用主成分分析进行姿态识别的方法，从CG图像库生成自由视点图像。为了更好地对运动目标进行分割和跟踪，还提出并发展了快速光照变化下运动目标的稳健分割和基于粒子滤波的多固定摄像机运动目标分割方法。这些结果为本课题的研究提供了良好的可能性。

This research aims the new possibility for computer vision and mixed reality that includes the following topics. The topics are Neural network implementation of 3-D modeling and virtual image generation, obtaining shape from shading images from Scanning Electron Microscope, Recognition of shape and generation of free view point image from image database, Extracting moving object with robust background subtraction for quick illumination change, and Improvement and efficient approach with particle filter based motion tracking approach under fixed view point. Neural network is used to the extension for color object image, where the color reflectance factor is recovered with shape information, in addition, virtual image with any light source direction and any view point is generated for the mixed reality. Developed NN based approach gives better reality in comparison with previous CG model based approaches. Recovering shape from SEM was developed with Hopfield neural network as the optimization approach. A new approach to generate free viewpoint image from CG database is also proposed using pose recognition with PCA from CG image database. For the purpose of better segmentation and tracking of moving object, robust segmentation of moving object under the quick illumination changes and particle filter based approach with multiple fixed cameras are also proposed and developed. These results could have good possibility for the research topics.