Multi-view computational scene modelling

近年来，我们目睹了3D重建精度的显著提高。随着低成本深度传感器和更健壮的多视点重建算法的出现，场景建模技术已经变得流行起来，不再局限于实验室环境。然而，尽管它们具有灵活性，但现有技术仍然局限于具有简单表面反射属性(主要是漫反射)的场景。虽然已经提出了更复杂的光度学方法来处理这类场景，但它们往往需要昂贵的专业设备，而这些设备无法部署在实验室之外。该项目将通过开发更通用和通用的数字化技术来填补这一空白，这些技术可以应用于使用消费者硬件捕获的具有复杂反射率的自然场景。基于计算机视觉和计算摄影的最新进展，该项目将引入新的采集硬件和算法，以推断深度以及其他重要属性，如表面法线和BRDF。为了进一步提高建模精度，研究还将考虑降低重建精度的复杂物理现象，如次表面散射和相互反射，并将探索如何使算法对这些影响更具弹性。这项研究的应用将在电影、游戏和文化遗产等创意行业的背景下进行演示，这些行业都需要创建资产的照片逼真的数字副本的能力。将特别强调对具有复杂表面反射率的自然或人造场景(例如，光泽材料)进行建模。预计这项技术将在许多其他需要精确建模的工业领域找到应用，例如在机器人或制造业中。

In recent years, we have witnessed dramatic improvements in 3D reconstruction accuracy. With the advent of low cost depth sensors and more robust multi-view reconstruction algorithms, scene modelling techniques have become popularised and no longer limited to lab environments. However, despite their flexibility, existing techniques remain limited to scenes with simple surface reflectance properties (mostly diffuse). While more complex photometric approaches have been proposed to deal with such scenes, they tend to require expensive specialist equipment that cannot be deployed outside of the lab.This project will fill this gap by developing more general and versatile digitisation techniques that can be applied to natural scenes with complex reflectance captured using consumer hardware. Building on recent advances in computer vision and computational photography, the project will introduce novel acquisition hardware and algorithms to infer depth together with other important attributes such as surface normal and BRDF. To further push the limits of modelling accuracy, the research will also consider complex physical phenomena which degrade reconstruction accuracy, such as sub-surface scattering and inter-reflections, and will explore how algorithms can be made more resilient to these effects.The application of this research will be demonstrated in the context of the creative industries such as film, gaming and cultural heritage which all require the ability to create photo-realistic digital copies of assets. A particular emphasis will be placed on modelling natural or man-made scenes which have a complex surface reflectance (e.g. glossy materials). It is anticipated that this technology will find applications in many other areas of industry which require accurate modelling such as in robotics or the manufacturing industry.