A Generalized and Flexible Deep Learning Framework for the Reconstruction of High Dynamic Range Images and Video

目前的图像和视频HDR重建方法存在许多局限性，我们的目标是在本文中解决其中的一些问题。目前最先进的方法具有严格的输入要求，不能自适应地利用任意数量的输入图像来重建HDR图像。2. 运动对齐仍然是一个具有挑战性的问题，特别是在HDR情况下，输入帧具有不同的曝光，并且可能面临诸如重噪声和大饱和区域等问题。3. HDR视频估计方法是计算密集型的，不适合像智能手机摄影这样的动态用例。为了解决上述问题，我们建议为HDR图像和视频重建开发一个通用的、灵活的深度学习框架。首先，我们的方法将能够使用任意数量的输入图像，通过使用来自集合神经网络的原理[15,1]，使我们能够结合单图像和多图像方法的优点。这样就可以根据拍摄的场景自适应地决定使用多少帧和哪个曝光值。当有极端运动或图像曝光良好时，单帧可能是最佳选择，而在动态范围非常大且运动较少的场景中可能需要更多帧。其次，我们将研究如何在帧之间执行精确的运动对齐，特别是在HDR情况下，考虑到大噪声或饱和区域的问题。目前的方法[6,13,11]通常采用通用的运动对齐模型，而没有考虑与HDR重建相关的具体挑战，因此成功有限。明确建模这些误差源将使我们能够有效地减少伪影和改善重建细节。最后，我们将探讨以高效的方式将我们的方法扩展到视频重建的可能性，以便在智能手机上实时捕获高分辨率HDR视频内容。将于2024年2月更新（三年级升学）

There are a number of limitations to current methods of HDR reconstruction for images and videos and we aim to address some of these in this thesis1. Currently state-of-the-art methods have rigid input requirements and cannot adaptively make use of an arbitrary number of input images to reconstruct HDR images. 2. Motion alignment remains a challenging problem, especially in the HDR case where the input frames have varying exposures and can face problems such as heavy noise and large saturated regions. 3. HDR video estimation methods are computationally intensive and unsuited to on-the-fly use cases such as smartphone photography. To address the issues outlined above, we propose to develop a generalized and flexible deep learning framework for HDR image and video reconstruction. Firstly, our method will be capable of using an arbitrary number of input images by using principles from set neural networks [15, 1], allowing us to combine the benefits of single image and multi-image approaches. This would make it possible to adaptively decide how many frames and which exposure values to use depending on the scene being captured. A single frame might be the best choice when there is extreme motion or the image is well-exposed, while more frames might be required in scenes with a very large dynamic range and less extreme motion. Secondly, we will investigate how to perform accurate motion alignment between frames specifically for the HDR case, taking into account the issues of large noisy or saturated regions. Current approaches [6, 13, 11] typically apply a generic motion alignment model without considering the specific challenges associated with HDR reconstruction and hence have limited success. Explicitly modelling these sources of error will allow us to effectively reduce ghosting artefacts and improve reconstruction detail. Finally, we will explore the possibility of extending our method to video reconstruction in a highly efficient manner for on-the-fly capture of high resolution HDR video content on smartphones. To be updated in February 2024 (at Year 3 Progression)