A Generalized and Flexible Deep Learning Framework for the Reconstruction of High Dynamic Range Images and Video

目前用于图像和视频的HDR重建方法存在一些限制，我们的目标是在本论文中解决其中的一些限制。目前，最先进的方法具有严格的输入要求，并且不能自适应地利用任意数量的输入图像来重建HDR图像。2.运动对齐仍然是一个具有挑战性的问题，特别是在HDR的情况下，输入帧具有不同的曝光，并且可能面临严重的噪声和大的饱和区域等问题。3.HDR视频估计方法计算量大，不适合智能手机拍照等即时使用案例。为了解决上述问题，我们提出了一种通用的、灵活的HDR图像和视频重建深度学习框架。首先，通过使用集合神经网络[15，1]的原理，我们的方法将能够使用任意数量的输入图像，允许我们结合单图像和多图像方法的优点。这将使其能够根据所捕捉的场景自适应地决定使用多少帧和哪些曝光值。当存在极端运动或图像曝光良好时，单帧可能是最佳选择，而在具有非常大的动态范围和较不极端运动的场景中，可能需要更多帧。其次，我们将研究如何针对HDR情况在帧之间执行精确的运动对齐，同时考虑到大噪声或饱和区域的问题。目前的方法[6，13，11]通常应用通用的运动对齐模型，而没有考虑与HDR重建相关的具体挑战，因此成功有限。明确地对这些误差源进行建模将使我们能够有效地减少重影伪影并改进重建细节。最后，我们将探索将我们的方法以高效的方式扩展到视频重建的可能性，以便在智能手机上动态捕获高分辨率HDR视频内容。将于2024年2月更新(第三年升级)

There are a number of limitations to current methods of HDR reconstruction for images and videos and we aim to address some of these in this thesis1. Currently state-of-the-art methods have rigid input requirements and cannot adaptively make use of an arbitrary number of input images to reconstruct HDR images. 2. Motion alignment remains a challenging problem, especially in the HDR case where the input frames have varying exposures and can face problems such as heavy noise and large saturated regions. 3. HDR video estimation methods are computationally intensive and unsuited to on-the-fly use cases such as smartphone photography. To address the issues outlined above, we propose to develop a generalized and flexible deep learning framework for HDR image and video reconstruction. Firstly, our method will be capable of using an arbitrary number of input images by using principles from set neural networks [15, 1], allowing us to combine the benefits of single image and multi-image approaches. This would make it possible to adaptively decide how many frames and which exposure values to use depending on the scene being captured. A single frame might be the best choice when there is extreme motion or the image is well-exposed, while more frames might be required in scenes with a very large dynamic range and less extreme motion. Secondly, we will investigate how to perform accurate motion alignment between frames specifically for the HDR case, taking into account the issues of large noisy or saturated regions. Current approaches [6, 13, 11] typically apply a generic motion alignment model without considering the specific challenges associated with HDR reconstruction and hence have limited success. Explicitly modelling these sources of error will allow us to effectively reduce ghosting artefacts and improve reconstruction detail. Finally, we will explore the possibility of extending our method to video reconstruction in a highly efficient manner for on-the-fly capture of high resolution HDR video content on smartphones. To be updated in February 2024 (at Year 3 Progression)