TPDNet: Texture-Guided Phase-to-DEPTH Networks to Repair Shadow-Induced Errors for Fringe Projection Profilometry
TPDNet: Texture-Guided Phase-to-DEPTH Networks to Repair Shadow-Induced Errors for Fringe Projection Profilometry
复制标题
DOI：
10.3390/photonics10030246
复制
发表时间：
2023-02
期刊：
影响因子：
2.4
通讯作者：
Jiaqiong Li;Beiwen Li
中科院分区：
文献类型：
--
作者：
Jiaqiong Li;Beiwen Li
本文提出了一种相位到深度的深度学习模型，用于修复条纹投影轮廓术（FPP）中阴影引起的误差。该模型包括两个沙漏分支，分别从纹理图像和相位图中提取信息，并通过级联和权重融合两个分支的信息。该模型的输入包含纹理图像、掩模和未包裹的相位图，而地面真实值是来自CAD模型的深度图。选择损失函数来考虑图像细节和结构相似性。在验证的虚拟FPP系统中，训练数据包含1200个样本。训练完成后，我们对虚拟和现实扫描数据进行了实验，结果支持了模型的有效性。验证数据集的平均绝对误差和均方根误差分别为1.0279 mm和1.1898 mm。此外，我们还分析了环境光强度对模型性能的影响。低环境光限制了模型的性能，因为模型不能从纹理图像中完全黑暗的阴影区域提取有效信息。研究了各分支网络的贡献。来自纹理主导分支的特征被用作纠正阴影引起的错误的指导。来自相位优势分支网络的信息可以对整个目标进行准确的预测。该模型为FPP系统中阴影误差的修复提供了很好的参考。
This paper proposes a phase-to-depth deep learning model to repair shadow-induced errors for fringe projection profilometry (FPP). The model comprises two hourglass branches that extract information from texture images and phase maps and fuses the information from the two branches by concatenation and weights. The input of the proposed model contains texture images, masks, and unwrapped phase maps, and the ground truth is the depth map from CAD models. A loss function was chosen to consider image details and structural similarity. The training data contain 1200 samples in the verified virtual FPP system. After training, we conduct experiments on the virtual and real-world scanning data, and the results support the model’s effectiveness. The mean absolute error and the root mean squared error are 1.0279 mm and 1.1898 mm on the validation dataset. In addition, we analyze the influence of ambient light intensity on the model’s performance. Low ambient light limits the model’s performance as the model cannot extract valid information from the completely dark shadow regions in texture images. The contribution of each branch network is also investigated. Features from the texture-dominant branch are leveraged as guidance to remedy shadow-induced errors. Information from the phase-dominant branch network makes accurate predictions for the whole object. Our model provides a good reference for repairing shadow-induced errors in the FPP system.