A scalable non-intrusive image annotation method using eye tracking for training deep learning models in radiology

项目概要/摘要 机器学习（ML）和阿尔蒂智能最近已经成为可以增强 放射学解释，并显示出改善患者预后的希望。ML的一种方法是 对医疗保健的重大影响是改善了对早期体征的高容量，低成本检查的评估 常规胸部X光检查是一个“筛查”疾病的机会，包括 癌症、慢性阻塞性肺病（COPD）、肺炎和充血性心力衰竭。比如说， 肺癌是美国癌症死亡的最常见原因，通常在较高阶段被诊断出来。 导致存活率较低。国家肺筛查试验报告说，低剂量 计算机断层扫描（LDCT）筛查导致肺癌死亡率降低20%;然而， 符合条件的人实际上接受LDCT筛查。与此同时，胸部X光仍然是最常见的形式 世界范围内的成像。改进的X射线检测可以指导患者进行LDCT。COPD是另一个重要的 这是一种经常被低估的疾病。COPD患者患肺癌和呼吸道疾病的风险增加 感染或恶化，这与较高的发病率和死亡率。此外，胸部X光检查 可能显示与肺炎有关的定义不清的实变区域。医疗护理是重- 需要治疗感染或评估其他原因。更普遍地说，通过胸部X光检查疾病的方法 可扩展到心脏肥大、肺水肿和胸腔积液，这些在充血性心脏中可见 失败改进的检测可以指导患者接受医疗护理。卷积神经网络（CNN），一种高度 成功的ML模型，可以应用于胸部X射线图像。然而，很少有带注释的医学数据集存在 足够大来训练CNN。此外，已经表明，用于定位的边界框 疾病可以被纳入CNN的训练中，并显著提高其准确性。不幸的是， 具有这种局部化注释的医疗数据集甚至更罕见， 到放射科医生创建边界框的耗时过程。我们提出了一个创新的综合 使用眼动跟踪、语音记录和新颖的视觉和语言模型来创建本地化注释的方法， 以非侵入放射科医师工作流程的方式进行。我们方法的新奇在于 常规放射学阅读时的眼动跟踪。挑战在于克服相对模糊的性质 与提供有关异常的明确信息的边界框相比，眼动跟踪信息。 为了应对这一挑战，我们还将设计新的CNN架构和学习算法， 跟踪和附加信息，例如瞳孔扩张和持续时间。所提出的方法可以 轻松扩展以创建非常大的数据集，而不会为放射科医生带来额外的工作量。此外，委员会认为， 部署在阅读室中，它可以提供连续的注释图像流以扩展训练集。

PROJECT SUMMARY/ABSTRACT Machine learning (ML) and artificial intelligence have recently emerged as powerful techniques that can augment radiology interpretations and show promise for improving patient outcomes. One of the ways for ML to make a significant impact on health care is in improving the evaluation of high-volume, low-cost exams for early signs of a wide variety of diseases.The routine chest x-ray is an ”opportunity for screening” for diseases, including cancer, chronic obstructive pulmonary disease (COPD), pneumonia and congestive heart failure. For instance, lung cancer is the most common cause of cancer death in the US, and is typically diagnosed at a higher stage than most other cancers leading to low survival rates. The National Lung Screening Trial reported that low dose computed tomography (LDCT) screening resulted in a 20% reduction in lung cancer mortality; however, few eli- gible people actually undergo LDCT screening. Meanwhile, chest x-rays continue to be the most common form of imaging worldwide. Improved detection from x-rays can direct patients to LDCT. COPD is another important disease that is often under-diagnosed. People with COPD are at increased risk of lung cancer and respiratory infections, or exacerbations, which are associated with higher morbidity and mortality. Furthermore, a chest x-ray may show poorly-defined regions of consolidation that are concerning for pneumonia. Medical attention is re- quired to treat an infection or evaluate for other cause. More generally, methods to detect disease on chest x-rays can be extended to cardiomegaly, pulmonary edema and pleural effusions which are seen in congestive heart failure. Improved detection can direct patients to medical care. Convolutional neural networks (CNN), a highly successful ML model, can be applied to chest x-ray images. However, few annotated medical datasets exist that are sufficiently large to train CNNs. Furthermore, it has been shown that bounding boxes used to localize disease can be incorporated into the training of CNNs and significantly increase their accuracy. Unfortunately, medical datasets with such localized annotations are even rarer and are very limited in the number of cases due to the time-consuming process of creating bounding boxes by radiologists. We propose an innovative integrated approach using eye tracking, speech recording and novel vision and language models to create localized annota- tions in a manner that is non-intrusive to the workflow of the radiologist. The novelty of our approach is in the use of eye tracking during routine radiological reading. The challenge is to overcome the relatively ambiguous nature of eye tracking information compared to bounding boxes which provide definitive information about abnormalities. To address this challenge, we will also design new CNN architectures and learning algorithms that can use eye tracking and additional information such as pupil dilation and fixation duration. The proposed methodology can easily scale up to create very large datasets without generating additional workload for radiologists. Furthermore, deployed in the reading room, it could provide a continuous stream of annotated images to expand training sets.