Analysis of behaviors of non-rigid objects in real space and their real-time reproduction in 3D virtual space

本计画的目的是建立分析非刚性物体在真实的空间中的行为并在3D虚拟空间中即时重现其行为的方法。作为研究对象，本项目重点研究了1.水，2。树木，3。人类的形象。1.我们澄清了凹凸贴图适用于在3D中对水建模。然后，研究了如何获取水面上多点的表面法线，其中凹凸贴图需要表面法线进行渲染。提出了一种基于“阴影恢复形状”（Shape from Shading，SFS）的方法。事实证明，SFS是有用的亮度变化平稳的地区，但形状不连续出现在周围的水的物体的反射图像的边界。我们提出了一种处理这个问题的方法，并得到了令人满意的实验结果.研究了如何高效逼真地生成三维植物模型的动画。我们的方法分析了每个分支的代表性分支段的运动，而不分析所有的分支段，其中3D运动是有效的。实验结果表明了该方法的有效性.人类行为分为面部表情和身体动作。关于面部表情，我们探索了一种将真实的面部图像纹理映射到再现面部肌肉结构和动作的3D面部模型的方法，结果证明可以再现皱纹等精确结构。关于身体动作，我们研究了一种基于颜色分析的方法，该方法估计穿着特殊套装的人的姿势，该套装在身体的每个部位都有不同的颜色，从而可以解决遮挡问题。此外，进行了识别人与物体之间的相互作用的初步研究。

The purpose of this project is to establish methods that analyze the behaviors of non-rigid objects in real space and reproduce the behaviors in 3D virtual space in real-time. As research targets, this project focuses on 1. water, 2. botanical trees, and 3. human images.1. We clarified that Bump mapping is appropriate for modeling water in 3D. Then, we studied how to acquire surface normals to many points in water surface, where Bump mapping needs the surface normals for rendering. We proposed a method based on "Shape from Shading" (SFS). It turned out that SFS is useful for areas in which the brightness changes smoothly, but shape discontinuities appear in the boundaries of the reflected images of the objects around the water. We proposed a method that deals with this issue, and we got promising experimental results.2. We studied how to animate 3D botanical tree models efficiently and realistically. Our approach analyzes the motion of the representative branch segment of each branch without analyzing all the branch segments, where the 3D motions are efficiently. Experimental results show the effectiveness of the proposed method.3. Human behaviors are classified into facial expressions and body actions. Concerning facial expressions, we explored a method that texture-maps real facial images to the 3D face model that reproduces the structure and actions of facial muscles, and it turned out that precise structures such as wrinkles can be reproduced. Concerning body actions, we studied a color analysis based method that estimates the postures of the person who wears a special suit that is colored differently in each body part so that occlusion problems can be solved. Furthermore, preliminary study of recognizing interactions between a human and object was conducted.