Realistic Shape from Shading

从明暗恢复形状(Shape-From-Shading，SFS)是计算机视觉中的经典问题。它的目的是通过一张照片中阴影的变化来估计3D表面的形状。它只用一张图像就能恢复形状的事实使SFS对广泛的应用程序具有吸引力，特别是当其他3D成像技术，如立体或深度扫描仪难以应用时。例如，可以在合成孔径雷达(SAR)图像的地形分析、医学图像重建、微电子检测、CAD系统和娱乐业中找到应用实例。然而，尽管经过了四十多年的研究，SFS仍然是一个具有挑战性的问题，由于缺乏稳健性，有时结果令人难以置信，在实际问题中没有得到充分利用。一个好的解决方案是紧迫和具有挑战性的。这个项目的目的是开发一个健壮和实用的SFS算法来从真实世界的图像中准确地恢复形状。目前SFS在真实世界图像上表现不佳的原因有几个潜在的原因。第一个问题是，传统的平行投影、朗伯反射和简单照明模型的假设对于真实世界的曲面是不准确的。第二个原因是SFS是一个约束不足的问题：人类视觉系统不仅从阴影中恢复形状，而且还从轮廓、阴影和先前的经验中恢复形状。在计算机视觉中，很少有工作考虑从阴影到形状与其他视觉线索和人类互动的结合。第三个也是很大程度上被忽视的原因是，许多真实的曲面并不光滑，并且具有详细的特征。现有的SFS算法大多只适用于光滑表面的图像，并且往往会过度平滑任何特征，基于这些观察，本项目将结合特征感知图像过滤、从线条绘制形状和用户交互等领域的技术，从复杂的真实图像中实现更准确的形状恢复。将为现实应用开发一个SFS互动平台。研究成果将在CAD和计算机视觉的各种应用上进行测试：具体地说，作为项目的一部分，我们将探索在基本地貌生成和人脸识别方面的应用。

Shape-from-shading (SFS) is a classic problem in computer vision. It aims to estimate 3D surface shape from the variations in shading in a single photographic image. The fact that it recovers shape using only a single image makes SFS attractive to a wide range of applications, especially when other 3D imaging techniques such as stereo or depth scanners are difficult to apply. Example applications can be found in topography analysis of SAR (synthetic aperture radar) images, reconstruction of medical images, inspection of microelectronics, CAD systems, and the entertainment industry. However, despite over four decades research, SFS still remains a challenging problem which is underused in real world problems, due to a lack of robustness, and sometimes implausible results. A good solution is pressing and challenging. This project intends to develop a robust and practical SFS algorithm for accurate shape recovery from real-world images.The reasons for SFS's current poor performance on real-world images have several underlying causes. The first is that the classic assumptions of orthographic projection, Lambertian reflection, and simple lighting models are inaccurate for real-world surfaces. The second reason is that SFS is an underconstrained problem: the human visual system recovers shape not only from shading, but also from outlines, shadows, and prior experience. In computer vision, little work has considered the combination of shape from shading with other visual cues and human interactions. The third and largely overlooked reason is that many real surfaces are not smooth, and have detailed features. Most existing SFS algorithms only apply to images of smooth surfaces, and tend to over-smooth any features.Based on these observations, this project will integrate techniques from such areas as feature-aware image filtering, shape from line drawing, and user interaction, to achieve more accurate shape recovery from sophisticated real-world images. An interactive platform for SFS will be developed for realistic applications. The outcome of the research will be tested on various applications in CAD and computer vision: specifically, as part of the project, we will explore the applications to bas-relief generation, and face recognition.