Deep Multi-Scale Mesh Feature Learning for Automated Labeling of Raw Dental Surfaces From 3D Intraoral Scanners
Deep Multi-Scale Mesh Feature Learning for Automated Labeling of Raw Dental Surfaces From 3D Intraoral Scanners
复制标题
DOI：
10.1109/tmi.2020.2971730
复制
发表时间：
2020-07-01
影响因子：
10.6
通讯作者：
Shen, Dinggang
中科院分区：
文献类型：
--
作者：
Lian, Chunfeng;Wang, Li;Shen, Dinggang
在数字化三维牙面模型上精确标记牙齿是正畸治疗计划中牙齿位置重排的前提。然而，这是一项具有挑战性的任务，主要是由于患者牙齿的异常和变化外观。口腔内扫描仪（IOS）在临床上的应用进一步增加了自动标记牙齿的难度，因为IOS获得的牙齿表面在牙龈和口腔深处通常质量较低。近年来，在计算机视觉和图形学领域提出了一些开创性的端到端方法（如PointNet），直接使用原始表面进行三维形状分割。尽管这些方法可能适用于我们的任务，但大多数方法都无法捕获细粒度的局部几何背景，而这对于识别具有不同形状和外观的小牙齿至关重要。在本文中，我们提出了一种端到端的深度学习方法，称为MeshSegNet，用于在原始牙齿表面上自动标记牙齿。使用多个原始表面属性作为输入，MeshSegNet沿着其前向路径集成了一系列图约束学习模块，以分层地提取多尺度局部上下文特征。然后，采用密集融合策略，结合局部到全局的几何特征，学习更高层次的特征，用于网格单元标注。我们的MeshSegNet产生的预测通过图形切割细化步骤进一步后处理，以进行最终分割。我们使用3D IOS获取的真实患者数据集（包括原始上颌表面）对MeshSegNet进行了评估。经过5次交叉验证的实验结果表明，MeshSegNet在3D形状分割方面明显优于最先进的深度学习方法。
Precisely labeling teeth on digitalized 3D dental surface models is the precondition for tooth position rearrangements in orthodontic treatment planning. However, it is a challenging task primarily due to the abnormal and varying appearance of patients' teeth. The emerging utilization of intraoral scanners (IOSs) in clinics further increases the difficulty in automated tooth labeling, as the raw surfaces acquired by IOS are typically low-quality at gingival and deep intraoral regions. In recent years, some pioneering end-to-end methods (e.g., PointNet) have been proposed in the communities of computer vision and graphics to consume directly raw surface for 3D shape segmentation. Although these methods are potentially applicable to our task, most of them fail to capture fine-grained local geometric context that is critical to the identification of small teeth with varying shapes and appearances. In this paper, we propose an end-to-end deep-learning method, called MeshSegNet, for automated tooth labeling on raw dental surfaces. Using multiple raw surface attributes as inputs, MeshSegNet integrates a series of graph-constrained learning modules along its forward path to hierarchically extract multi-scale local contextual features. Then, a dense fusion strategy is applied to combine local-to-global geometric features for the learning of higher-level features for mesh cell annotation. The predictions produced by our MeshSegNet are further post-processed by a graph-cut refinement step for final segmentation. We evaluated MeshSegNet using a real-patient dataset consisting of raw maxillary surfaces acquired by 3D IOS. Experimental results, performed 5-fold cross-validation, demonstrate that MeshSegNet significantly outperforms state-of-the-art deep learning methods for 3D shape segmentation.