Test-Time Training Can Close the Natural Distribution Shift Performance Gap in Deep Learning Based Compressed Sensing
Test-Time Training Can Close the Natural Distribution Shift Performance Gap in Deep Learning Based Compressed Sensing
复制标题
DOI：
--
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
Mohammad Zalbagi Darestani;Jiayu Liu;Reinhard Heckel
中科院分区：
文献类型：
--
作者：
Mohammad Zalbagi Darestani;Jiayu Liu;Reinhard Heckel
基于深度学习的图像重建方法优于传统方法。然而，当应用于来自与训练图像不同的分布的图像时，神经网络遭受性能下降。例如，在加速磁共振成像（MRI）中训练用于重建膝盖的模型不能很好地重建大脑，即使在大脑上训练的相同网络可以很好地重建大脑。因此，对于给定的神经网络，存在分布偏移性能差距，定义为在分布$P$上训练和在另一个分布$Q$上训练时的性能差异，并在$Q$上评估两个模型。在这项工作中，我们提出了一种基于深度学习的压缩感知域自适应方法，该方法依赖于训练期间的自我监督，并在推理时进行测试时训练。我们表明，对于四个自然分布的变化，这种方法基本上关闭了最先进的架构加速MRI的分布偏移性能差距。
Deep learning based image reconstruction methods outperform traditional methods. However, neural networks suffer from a performance drop when applied to images from a different distribution than the training images. For example, a model trained for reconstructing knees in accelerated magnetic resonance imaging (MRI) does not reconstruct brains well, even though the same network trained on brains reconstructs brains perfectly well. Thus there is a distribution shift performance gap for a given neural network, defined as the difference in performance when training on a distribution $P$ and training on another distribution $Q$, and evaluating both models on $Q$. In this work, we propose a domain adaptation method for deep learning based compressive sensing that relies on self-supervision during training paired with test-time training at inference. We show that for four natural distribution shifts, this method essentially closes the distribution shift performance gap for state-of-the-art architectures for accelerated MRI.