Modeling visual search behavior of breast radiologists using a deep convolution neural network
Modeling visual search behavior of breast radiologists using a deep convolution neural network
复制标题
DOI：
10.1117/1.jmi.5.3.035502
复制
发表时间：
2018-07-01
影响因子：
2.4
通讯作者：
Mello-Thoms, Claudia
中科院分区：
文献类型：
--
作者：
Mall, Suneeta;Brennan, Patrick C.;Mello-Thoms, Claudia
视觉搜索，使用眼球运动（扫视）和中央凹视觉检测和识别物体的过程，已经被研究用于识别乳房X线照片解释错误的根本原因。本研究的目的是使用深度机器学习方法对放射科医生的视觉搜索行为及其对乳房X线照片的解释进行建模。我们的模型基于深度卷积神经网络，这是一种受生物启发的多层感知器，它模拟视觉皮层，并通过迁移学习技术得到加强。眼动跟踪数据是从8名放射科医生（在阅读乳房X线照片方面具有不同的经验水平）中获得的，他们回顾了120个双视图数字乳房X线摄影病例（59种癌症），并已用于训练模型，该模型已使用ImageNet数据集进行预训练以进行迁移学习。从放射科医生的视觉搜索图（通过头戴式眼动跟踪设备获得）中提取接受直接（中心凹注视）、间接（外周注视）或无（从不注视）视觉关注的乳房X线照片区域。这些区域沿着放射科医师对疑似恶性肿瘤的存在的评估（包括评估中的置信度）用于建模：（1）放射科医师的决定，（2）放射科医师对这些决定的置信度，以及（3）注意力水平（即，中心凹、外周或无）。我们的研究结果表明，在建模这种行为的准确性和低误分类。(C)2018年，摄影光学仪器工程师协会（SPIE）
Visual search, the process of detecting and identifying objects using eye movements (saccades) and foveal vision, has been studied for identification of root causes of errors in the interpretation of mammograms. The aim of this study is to model visual search behavior of radiologists and their interpretation of mammograms using deep machine learning approaches. Our model is based on a deep convolutional neural network, a biologically inspired multilayer perceptron that simulates the visual cortex and is reinforced with transfer learning techniques. Eye-tracking data were obtained from eight radiologists (of varying experience levels in reading mammograms) reviewing 120 two-view digital mammography cases (59 cancers), and it has been used to train the model, which was pretrained with the ImageNet dataset for transfer learning. Areas of the mammogram that received direct (foveally fixated), indirect (peripherally fixated), or no (never fixated) visual attention were extracted from radiologists' visual search maps (obtained by a head mounted eye-tracking device). These areas along with the radiologists' assessment (including confidence in the assessment) of the presence of suspected malignancy were used to model: (1) radiologists' decision, (2) radiologists' confidence in such decisions, and (3) the attentional level (i.e., foveal, peripheral, or none) in an area of the mammogram. Our results indicate high accuracy and low misclassification in modeling such behaviors. (C) 2018 Society of Photo-Optical Instrumentation Engineers (SPIE)