Seven-Point Checklist and Skin Lesion Classification Using Multitask Multimodal Neural Nets
Seven-Point Checklist and Skin Lesion Classification Using Multitask Multimodal Neural Nets
复制标题
DOI：
10.1109/jbhi.2018.2824327
复制
发表时间：
2019-03-01
影响因子：
7.7
通讯作者：
Hamarneh, Ghassan
中科院分区：
文献类型：
--
作者：
Kawahara, Jeremy;Daneshvar, Sara;Hamarneh, Ghassan
我们提出了一种多任务深度卷积神经网络，在多模态数据（临床和皮肤镜图像以及患者元数据）上进行训练，以对7点黑色素瘤检查表标准进行分类并进行皮肤病变诊断。我们的神经网络使用多个多任务损失函数进行训练，其中每个损失都考虑了输入模态的不同组合，这使得我们的模型在推理时对缺失数据具有鲁棒性。我们的最终模型分类的7点检查表和皮肤状况诊断，产生多模态特征向量适合图像检索，并定位临床判别区域。我们使用1011例病变病例对我们的方法进行基准测试，并报告了所有7点标准和诊断的综合结果。
We propose a multitask deep convolutional neural network, trained on multimodal data (clinical and dermoscopic images, and patient metadata), to classify the 7-point melanoma checklist criteria and perform skin lesion diagnosis. Our neural network is trained using several multitask loss functions, where each loss considers different combinations of the input modalities, which allows our model to be robust to missing data at inference time. Our final model classifies the 7-point checklist and skin condition diagnosis, produces multimodal feature vectors suitable for image retrieval, and localizes clinically discriminant regions. We benchmark our approach using 1011 lesion cases, and report comprehensive results over all 7-point criteria and diagnosis.