A scalable non-intrusive image annotation method using eye tracking for training deep learning models in radiology

项目总结/文摘

PROJECT SUMMARY/ABSTRACT Machine learning (ML) and artificial intelligence have recently emerged as powerful techniques that can augment radiology interpretations and show promise for improving patient outcomes. One of the ways for ML to make a significant impact on health care is in improving the evaluation of high-volume, low-cost exams for early signs of a wide variety of diseases.The routine chest x-ray is an ”opportunity for screening” for diseases, including cancer, chronic obstructive pulmonary disease (COPD), pneumonia and congestive heart failure. For instance, lung cancer is the most common cause of cancer death in the US, and is typically diagnosed at a higher stage than most other cancers leading to low survival rates. The National Lung Screening Trial reported that low dose computed tomography (LDCT) screening resulted in a 20% reduction in lung cancer mortality; however, few eli- gible people actually undergo LDCT screening. Meanwhile, chest x-rays continue to be the most common form of imaging worldwide. Improved detection from x-rays can direct patients to LDCT. COPD is another important disease that is often under-diagnosed. People with COPD are at increased risk of lung cancer and respiratory infections, or exacerbations, which are associated with higher morbidity and mortality. Furthermore, a chest x-ray may show poorly-defined regions of consolidation that are concerning for pneumonia. Medical attention is re- quired to treat an infection or evaluate for other cause. More generally, methods to detect disease on chest x-rays can be extended to cardiomegaly, pulmonary edema and pleural effusions which are seen in congestive heart failure. Improved detection can direct patients to medical care. Convolutional neural networks (CNN), a highly successful ML model, can be applied to chest x-ray images. However, few annotated medical datasets exist that are sufficiently large to train CNNs. Furthermore, it has been shown that bounding boxes used to localize disease can be incorporated into the training of CNNs and significantly increase their accuracy. Unfortunately, medical datasets with such localized annotations are even rarer and are very limited in the number of cases due to the time-consuming process of creating bounding boxes by radiologists. We propose an innovative integrated approach using eye tracking, speech recording and novel vision and language models to create localized annota- tions in a manner that is non-intrusive to the workflow of the radiologist. The novelty of our approach is in the use of eye tracking during routine radiological reading. The challenge is to overcome the relatively ambiguous nature of eye tracking information compared to bounding boxes which provide definitive information about abnormalities. To address this challenge, we will also design new CNN architectures and learning algorithms that can use eye tracking and additional information such as pupil dilation and fixation duration. The proposed methodology can easily scale up to create very large datasets without generating additional workload for radiologists. Furthermore, deployed in the reading room, it could provide a continuous stream of annotated images to expand training sets.