High Quality 3D Geometry and Appearance Reconstruction of Non-Rigidly Deforming Objects using Low-Cost RGB-D Cameras

点击翻译按钮获取中文摘要

Capturing and reconstruction of high quality 3D geometry and the appearance of non-rigidly deforming objects, such as the dynamics of human actions, is essential for many applications, including movie and game production in the creative industries, Virtual Reality (VR) videoconferencing, analysing human behaviour for healthcare monitoring and sports analysis etc. Despite great effort, it is still a challenging problem, especially when large-scale deformations are involved: self-occlusion, subtle geometry and appearance change (e.g. wrinkles of skin and clothing) all contribute to the difficulty. This research aims to advance the state of the art by investigating novel data-driven techniques to address fundamental challenges. Low-cost RGB-Depth cameras have become more capable in recent years and will be used in the research to make the techniques widely useful. We will develop a new joint representation and analysis technique for both geometry and appearance, to effectively encode geometric and appearance change during non-rigid deformation. The plausible deformation and change in appearance typically form a low dimensional manifold embedded in this joint space. To address the issues of noise and incompleteness in the scanned data, machine learning techniques such as manifold learning will be exploited. This will effectively utilise information from any previous scans to fill the gaps and improve the quality of reconstruction. An optimisation framework will also be developed incorporating knowledge from the manifold as well as sparse priors.To make the research feasible, the project is built on top of the supervisors' existing work on shape deformation (representation and shape space analysis), non-rigid registration, data-driven reconstruction and sparse models, all of which have recent publications in top journals. The project is related and complementary to two current Royal Society international collaborative projects with leading research institutions in China. The research and the PhD student will benefit greatly from these collaborations.