Thyroid nodule recognition using a joint convolutional neural network with information fusion of ultrasound images and radiofrequency data
Thyroid nodule recognition using a joint convolutional neural network with information fusion of ultrasound images and radiofrequency data
复制标题
使用超声图像和射频数据信息融合的联合卷积神经网络进行甲状腺结节识别
DOI：
10.1007/s00330-020-07585-z
复制
发表时间：
2021-01-06
影响因子：
5.9
通讯作者：
Zou, Ruhai
中科院分区：
文献类型：
--
作者：
Liu, Zhong;Zhong, Shaobin;Zou, Ruhai
目的提出一种基于深度学习的超声图像与射频信号信息融合的方法，用于甲状腺结节(TNS)的分类。方法对163对成人TNS的超声图像和射频信号进行分析。我们开发了一种基于信息融合的联合卷积神经网络(IF-JCNN)，用于鉴别诊断恶性和良性TNS。IF-JCNN包含两个用于深度特征提取的分支CNN：一个用于US图像，另一个用于RF信号。提取的特征在IF-JCNN的后端进行融合，用于TN分类。结果通过5次交叉验证，以超声图像和射频信号为输入的IF-JCNN对TN分类的准确性、敏感性、特异性和受试者工作特征曲线下面积分别为0.896(95%CI 0.838~0.938)、0.885(95%CI 0.804~0.941)、0.910(95%CI 0.815~0.966)和0.956(95%CI 0.926~0.987)，这比仅使用US图像获得的结果更好：0.822(0.755-0.878；P=0.0044)，0.792(0.679-0.868，p=0.0091)，0.866(0.760-0.937，p=0.197)和0.901(0.855-0.948，p=.0398)，或射频信号：0.767(0.694-0.829，p<；0.001)，0.781(0.685-0.859，p=0.0037)，0.746(0.625-0.845，p<；0.001)，0.845(0.786-0.903，宝洁0.001)。结论IF-JCNN模型填补了单纯利用CNN超声图像定性TNS的空白，有望成为一种辅助诊断甲状腺癌的工具。
Objective To develop a deep learning-based method with information fusion of US images and RF signals for better classification of thyroid nodules (TNs). Methods One hundred sixty-three pairs of US images and RF signals of TNs from a cohort of adult patients were used for analysis. We developed an information fusion-based joint convolutional neural network (IF-JCNN) for the differential diagnosis of malignant and benign TNs. The IF-JCNN contains two branched CNNs for deep feature extraction: one for US images and the other one for RF signals. The extracted features are fused at the backend of IF-JCNN for TN classification. Results Across 5-fold cross-validation, the accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) obtained by using the IF-JCNN with both US images and RF signals as inputs for TN classification were respectively 0.896 (95% CI 0.838-0.938), 0.885 (95% CI 0.804-0.941), 0.910 (95% CI 0.815-0.966), and 0.956 (95% CI 0.926-0.987), which were better than those obtained by using only US images: 0.822 (0.755-0.878; p = 0.0044), 0.792 (0.679-0.868, p = 0.0091), 0.866 (0.760-0.937, p = 0.197), and 0.901 (0.855-0.948, p = .0398), or RF signals: 0.767 (0.694-0.829, p < 0.001), 0.781 (0.685-0.859, p = 0.0037), 0.746 (0.625-0.845, p < 0.001), 0.845 (0.786-0.903, p < 0.001). Conclusions The proposed IF-JCNN model filled the gap of just using US images in CNNs to characterize TNs, and it may serve as a promising tool for assisting the diagnosis of thyroid cancer.