Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations
Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations
复制标题
DOI：
10.1109/cvpr52688.2022.01826
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Aishik Konwer;Xuan Xu;Joseph Bae;Chaoyu Chen;P. Prasanna
中科院分区：
文献类型：
--
作者：
Aishik Konwer;Xuan Xu;Joseph Bae;Chaoyu Chen;P. Prasanna
从医学图像预测临床结果或严重程度在很大程度上集中在从单一时间点或快照扫描学习表示。已有研究表明，时间成像可以更好地描述疾病进展的特征。因此，我们假设，通过利用序列图像中的疾病进展信息，可以改善结果预测。我们提出了一种深度学习方法，该方法利用时间进程信息来改善来自单个时间点图像的临床结果预测。在我们的方法中，基于自我注意的时间卷积网络(TCN)被用来学习最能反映疾病轨迹的表示。同时，视觉转换器以自监督的方式进行预训练，以从单时间点图像中提取特征。主要贡献是设计了一个重新校准模块，该模块使用最大平均差异损失(MMD)来对齐上述两种上下文表示的分布。我们训练我们的系统从单个时间点图像预测临床结果和严重程度等级。在胸部和骨关节炎放射成像数据集上的实验表明，我们的方法优于其他最先进的技术。
Clinical outcome or severity prediction from medical images has largely focused on learning representations from single-timepoint or snapshot scans. It has been shown that disease progression can be better characterized by temporal imaging. We therefore hypothesized that outcome predictions can be improved by utilizing the disease progression informationfrom sequential images. We present a deep learning approach that leverages temporal progression information to improve clinical outcome predictions from single-timepoint images. In our method, a self-attention based Temporal Convolutional Network (TCN) is used to learn a representation that is most reflective of the disease trajectory. Meanwhile, a Vision Transformer is pretrained in a self-supervised fashion to extract features from single-timepoint images. The key contribution is to design a recalibration module that employs maximum mean discrepancy loss (MMD) to align distributions of the above two contextual representations. We train our system to predict clinical outcomes and severity grades from single-timepoint images. Experiments on chest and osteoarthritis radiography datasets demonstrate that our approach outperforms other state-of-the-art techniques.