Automatic creation of annotations for chest radiographs based on the positional information extracted from radiographic image reports
Automatic creation of annotations for chest radiographs based on the positional information extracted from radiographic image reports
复制标题
根据从放射线图像报告中提取的位置信息自动创建胸部放射线照片注释
DOI：
10.1016/j.cmpb.2021.106331
复制
发表时间：
2021
影响因子：
6.1
通讯作者：
Matsumura Y.
中科院分区：
文献类型：
--
作者：
Wang B;Takeda T;Sugimoto K;Zhang J;Wada S;Konishi S;Manabe S;Okada K;Matsumura Y.
背景与目的本研究试图建立一种机器学习方法，利用提取的胸部X射线（CXR）报告信息标注的数据集，从CXR图像中检测疾病病变。我们将结节设定为靶病变。手动注释结节在时间方面是昂贵的。因此，我们使用的报告信息，自动产生训练数据的对象检测task.MethodsFirst，我们使用语义分割模型PSP-Net识别肺字段中描述的CXR报告。接下来，使用分类模型ResNeSt-50来区分分割的右场和左场中的结节。它还可以通过Grad-Cam提供注意力地图。如果关注区域对应于CXR报告中的结节位置，则生成关注边界框。最后，使用生成的注意边界框执行目标检测模型Faster-RCNN。对Faster-RCNN预测的边界框进行过滤，以满足从CXR报告中提取的位置。结果对于肺野分割，在我们的最佳模型中实现了0.889的联合平均交集。15，156张胸片用于分类。左肺和右肺的受试者工作特征曲线下面积分别为0.843和0.852。生成的注意力边界框的检测精度为0.341至0.531，这取决于注意力地图的二进制设置。通过目标检测过程，边界框的检测精度提高到0.567 ~ 0.800。结论基于CXR报告中提取的疾病位置信息，成功地在CXR图像上生成了包含结节的边界框。我们的方法有可能为各种肺部病变提供边界框，这可以减少专家的注释负担。Short abstractMachine learning for computer aided image diagnosis requires annotation of images，但手动注释对于医生来说是耗时的。在这项研究中，我们试图创建一种机器学习方法，该方法使用从CXR报告中提取的位置信息在胸部X射线（CXR）图像上创建具有疾病病变的边界框。我们将结节作为靶病变。首先，我们使用PSP网络分割肺野根据CXR报告。然后，使用分类模型ResNeSt-50来区分分割的肺野中的结节。我们还使用Grad-Cam算法创建了一个注意力地图。如果关注区域与CXR报告注释的区域相匹配，则边界框的坐标被视为可能的结节区域。最后，我们使用从结节分类模型中获得的注意力信息，并让对象检测模型由所有生成的边界框进行训练。通过目标检测模型，提高了包围盒检测结节的精度。
Background and objectiveIn this study, we tried to create a machine-learning method that detects disease lesions from chest X-ray (CXR) images using a data set annotated with extracted CXR reports information. We set the nodule as the target disease lesion. Manually annotating nodules is costly in terms of time. Therefore, we used the report information to automatically produce training data for the object detection task.MethodsFirst, we use semantic segmentation model PSP-Net to recognize lung fields described in the CXR reports. Next, a classification model ResNeSt-50 is used to discriminate the nodule in segmented right and left field. It also can provide attention map by Grad-Cam. If the attention region corresponds to the location of the nodule in the CXR reports, an attention bounding box is generated. Finally, object detection model Faster-RCNN was performed using generated attention bounding box. The bounding boxes predicted by Faster-RCNN were filtered to satisfy the location extracted from CXR reports.ResultsFor lung field segmentation, a mean intersection of union of 0.889 was achieved in our best model. 15,156 chest radiographs are used for classification. The area under the receiver operating characteristics curve was 0.843 and 0.852 for the left and right lung, respectively. The detection precision of the generated attention bounding box was 0.341 to 0.531 depending on the binary setting for attention map. Through object detection process, the detection precisions of the bounding boxes were improved to 0.567 to 0.800.ConclusionWe successfully generated bounding boxes with nodule on CXR images based on the positional information of the diseases extracted from the CXR reports. Our method has the potential to provide bounding boxes for various lung lesions which can reduce the annotation burden for specialists.Short abstractMachine learning for computer aided image diagnosis requires annotation of images, but manual annotation is time-consuming for medical doctor. In this study, we tried to create a machine-learning method that creates bounding boxes with disease lesions on chest X-ray (CXR) images using the positional information extracted from CXR reports. We set the nodule as the target lesion. First, we use PSP-Net to segment the lung field according to the CXR reports. Next, a classification model ResNeSt-50 was used to discriminate the nodule in segmented lung field. We also created an attention map using the Grad-Cam algorithm. If the area of attention matched the area annotated by the CXR report, the coordinate of the bounding box was considered as a possible nodule area. Finally, we used the attention information obtained from the nodule classification model and let the object detection model trained by all of the generated bounding boxes. Through object detection model, the precision of the bounding boxes to detect nodule is improved.