面向实际降质的Raw视频复原研究

近年来，基于深度学习的图像视频复原技术取得了显著进展。然而，目前的视频复原算法大多在sRGB域仿真的降质数据集上进行训练，难以处理实际成像系统采集的降质数据。相较于sRGB数据，传感器直出的Raw数据与拍摄场景的光度信息呈线性响应，保留了更丰富的原始信息，更有利于降质因素的建模。鉴于此，本项目研究面向实际降质的Raw视频复原算法，旨在获得高质量的Raw及sRGB视频重建。主要内容包括：研究Raw视频的降质模型，采用定格动画法采集真实的降质-优质视频数据集；探索渐进式的空间自适应特征重组方法，以高效实现相邻帧的空间对齐；鉴于不同设备采集的真实数据降质分布具有差异性，探索基于内部先验的特征调整机制以提升模型的自适应能力；以此为基础，针对视频去噪与超分辨率问题的特点，设计相应的网络结构，以期实现高质量的重建。本项目的视频复原理论与方法可以推广到多种视频处理问题，为智能视频处理提供新思路与新方法。

In recent years, the deep learning based image and video restoration has achieved great improvements. However, the current deep video restoration models are mostly trained with synthesized degraded sRGB videos, which cannot deal with the degraded videos captured by real imaging systems. Compared with sRGB videos, the raw videos, which are directly produced by the sensors, are linear with the aperture lighting and have more original imaging information, leading to easier modeling of the degradations. Therefore, the project explores raw video restoration towards realistic degradations, aiming at generating high quality raw and sRGB videos. The main research plan includes: 1) Investigate the degradation models in raw domain and utilize the stop-motion manner to capture the degraded and high-quality video pairs; 2) Investigate the progressively spatial-adaptive feature rearrangements to effectively align neighboring frames; 3) Considering that the degradation distributions are different among the videos captured by different devices, investigate the internal prior based feature adjustment to improve the domain adaption ability of trained models; 4) According to the characteristics of video denoising and super-resolution, construct the convolutional neural networks for the two tasks respectively to achieve high-quality reconstruction. The proposed video restoration method can be extended to many types of video processing problems (such as video dehazing, video enhancement, and video demoireing) and provide a novel idea and a new path for intelligent video processing.