Fine-grained spatial information extraction for radiology reports

摘要 近年来，自动生物医学图像分类在性能上有了巨大的改进， 尤其是在放射学方面。然而，实现这一点的机器学习(ML)方法令人惊叹 性能通常需要大量的标签数据来进行训练。一种越来越被接受的方式 通过对关联的自由文本报告使用自然语言处理(NLP)来获取这些数据 以一张图片为例，请看下面的脑部MRI报告片段： 有证据表明左侧顶叶脑软化与已知的既往中风病史一致。小的 左侧脑室外侧缘含铁血黄素沉积的灶性区域。 在这里，相关的MRI可以标记为脑软化症和含铁血黄素。NLP方法以 以这种方式自动标注影像已用于创建多个大型影像分类数据集 然而，如本例所示，放射学报告通常包含比 以前的NLP方法试图提取。上面例子中的两个发现都提到了它们的解剖位置， 这在语言学上被称为空间基础，因为位置将发现锚定在空间参考中。 此外，脑软化症的发现与中风的相关诊断有关，而含铁血黄素 FINDING提供了一种形态描述(小病灶)。这种细粒度的信息对图像很重要 分类，因为高级深度学习方法能够利用高粒度的结构化数据。这 这是合乎逻辑的，比如肺肿瘤的表现与肝脏肿瘤略有不同。如果一个最大似然算法 既能利用粗略的信息(肿瘤的一般表现)，又能识别细微的 粒度差异，它可以在特殊性和普适性之间找到最佳平衡。 从成像的角度来看，这也可以被视为图像级标签(这是 价格便宜，但需要大量数据用于训练-典型的数据集有数千张或更多的图像)和 分段(获取成本较高，但提供更好的训练数据-一个典型的数据集有40到200个 图像)，因为细粒度的空间标签对应于自然解剖段。 我们在这个项目中的基本假设是，如果可以从放射学报告中提取细粒度信息 使用NLP，当在足够大的图像上训练时，这将改善下游放射图像分类 数据集。对于放射学，粒度的主要形式是空间(位置、形状、方向等)，因此这将 成为我们努力的重点。我们进一步假设，这些NLP技术将可推广到大多数类型 放射学报告。然而，对于这个R21规模的项目，我们将重点介绍三种不同类型的 面临不同挑战的报告：胸部X光(研究最多、规模最大的图像分类之一 类型)，四肢X光(提供与胸部X光不同的结果)，以及大脑核磁共振(呈现 不同的图像形态和三维的额外复杂性)。

ABSTRACT Automated biomedical image classification has seen enormous improvements in performance over recent years, particularly in radiology. However, the machine learning (ML) methods that have achieved this remarkable performance often require enormous amounts of labeled data for training. An increasingly accepted means of acquiring this data is through the use of natural language processing (NLP) on the free-text reports associated with an image For example, take the following brain MRI report snippet: There is evidence of left parietal encephalomalacia consistent with known history of prior stroke. Small focal area of hemosiderin deposition along the lateral margins of the left lateral ventricle. Here, the associated MRI could be labeled for both Encephalomalacia and Hemosiderin. NLP methods to automatically label images in this way have been used to create several large image classification datasets However, as this example demonstrates, radiology reports often contain far more granular information than prior NLP methods attempted to extract. Both findings in the above example mention their anatomical location, which linguistically is referred to as a spatial grounding, as the location anchors the finding in a spatial reference. Further, the encephalomalacia finding is connected to the related diagnosis of stroke, while the hemosiderin finding provides a morphological description (small focal area). This granular information is important for image classification, as advanced deep learning methods are capable of utilizing highly granular structured data. This is logical, as for instance a lung tumor has a slightly different presentation than a liver tumor. If an ML algorithm can leverage both the coarse information (the general presentation of a tumor) while also recognizing the subtle granular differences, it can find an optimal balance between specificity and generalizability. From an imaging perspective, this can also be seen as a middle ground between image-level labels (which are cheap but require significant data for training—a typical dataset has thousands of images or more) and segmentation (which is expensive to obtain, but provides better training data—a typical dataset has 40 to 200 images), as the fine-grained spatial labels correspond to natural anatomical segments. Our fundamental hypothesis in this project is that if granular information can be extracted from radiology reports with NLP, this will improve downstream radiological image classification when training on a sufficiently large dataset. For radiology, the primary form of granularity is spatial (location, shape, orientation, etc.), so this will be the focus of our efforts. We further hypothesize that these NLP techniques will be generalizable to most types of radiology reports. For the purpose of this R21-scale project, however, we will focus on three distinct types of reports with different challenges: chest X-rays (one of the most-studied and largest-scale image classification types), extremity X-rays (which offer different findings than chest X-rays), and brain MRIs (which present a different image modality and the additional complexity of three dimensions).