Multi-Task Joint Learning Model for Segmenting and Classifying Tongue Images Using a Deep Neural Network
Multi-Task Joint Learning Model for Segmenting and Classifying Tongue Images Using a Deep Neural Network
复制标题
使用深度神经网络对舌头图像进行分割和分类的多任务联合学习模型
DOI：
10.1109/jbhi.2020.2986376
复制
发表时间：
2020-09-01
影响因子：
7.7
通讯作者：
Guo, Jinhong
中科院分区：
文献类型：
--
作者：
Xu, Qiang;Zeng, Yu;Guo, Jinhong
舌形图像自动分割和舌形图像分类是中医舌形特征的两大关键任务。由于舌图像分割的复杂性和舌图像分类的细粒度特征，这两项任务都具有挑战性。幸运的是，从计算机视觉的角度来看，这两个任务是高度相关的，这使得它们与多任务联合学习（Multi-Task Joint learning， MTL）的思想是兼容的。本文通过共享底层参数和添加两个不同的任务损失函数，提出了一种基于MTL的舌头图像分割与分类方法。此外，两种最先进的深度神经网络变体（UNET和判别过滤学习（DFL））被融合到MTL中来执行这两项任务。据我们所知，我们的方法是第一次尝试用MTL同时管理这两个任务。我们用提出的方法进行了大量的实验。实验结果表明，我们的联合方法优于现有的舌头表征方法。此外，提供了可视化和消融研究来帮助理解我们的方法，这表明我们的方法与人类的感知高度一致。
Automatic tongue image segmentation and tongue image classification are two crucial tongue characterization tasks in traditional Chinese medicine (TCM). Due to the complexity of tongue segmentation and fine-grained traits of tongue image classification, both tasks are challenging. Fortunately, from the perspective of computer vision, these two tasks are highly interrelated, making them compatible with the idea of Multi-Task Joint learning (MTL). By sharing the underlying parameters and adding two different task loss functions, an MTL method for segmenting and classifying tongue images is proposed in this paper. Moreover, two state-of-the-art deep neural network variants (UNET and Discriminative Filter Learning (DFL)) are fused into the MTL to perform these two tasks. To the best of our knowledge, our method is the first attempt to manage both tasks simultaneously with MTL. We conducted extensive experiments with the proposed method. The experimental results show that our joint method outperforms the existing tongue characterization methods. Besides, visualizations and ablation studies are provided to aid in understanding our approach, which suggest that our method is highly consistent with human perception.