Unsupervised Deep Video Denoising
Unsupervised Deep Video Denoising
复制标题
DOI：
10.1109/iccv48922.2021.00178
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
D. Y. Sheth;S. Mohan;Joshua L. Vincent;R. Manzorro;P. Crozier;Mitesh M. Khapra;Eero P. Simoncelli
中科院分区：
文献类型：
--
作者：
D. Y. Sheth;S. Mohan;Joshua L. Vincent;R. Manzorro;P. Crozier;Mitesh M. Khapra;Eero P. Simoncelli
深度卷积神经网络(CNN)用于视频去噪，通常是在有监督的情况下训练的，假设有干净的视频可用。然而，在许多应用中，如显微镜，无声视频是不可用的。为了解决这个问题，我们提出了一种无监督深度视频去噪器(UDVD1)，这是一种专门针对噪声数据进行训练的CNN架构。UDVD的性能可以与受监督的最先进水平相媲美，即使只在一段嘈杂的短视频上进行培训。我们通过对原始视频、荧光显微镜和电子显微镜数据进行去噪，展示了我们的方法在真实世界成像应用中的前景。与当前许多视频去噪方法不同，UDVD不需要显式的运动补偿。这是有利的，因为运动补偿在计算上是昂贵的，并且在输入数据有噪声时可能不可靠。基于梯度的分析表明，UDVD自动适应输入噪声视频中的局部运动。因此，该网络学会了执行隐式运动补偿，即使它只被训练用于去噪。
Deep convolutional neural networks (CNNs) for video denoising are typically trained with supervision, assuming the availability of clean videos. However, in many applications, such as microscopy, noiseless videos are not available. To address this, we propose an Unsupervised Deep Video Denoiser (UDVD1), a CNN architecture designed to be trained exclusively with noisy data. The performance of UDVD is comparable to the supervised state-of-the-art, even when trained only on a single short noisy video. We demonstrate the promise of our approach in real-world imaging applications by denoising raw video, fluorescence-microscopy and electron-microscopy data. In contrast to many current approaches to video denoising, UDVD does not require explicit motion compensation. This is advantageous because motion compensation is computationally expensive, and can be unreliable when the input data are noisy. A gradient-based analysis reveals that UDVD automatically adapts to local motion in the input noisy videos. Thus, the network learns to perform implicit motion compensation, even though it is only trained for denoising.