多聚焦视频的融合与编码研究

多聚焦视频作为一种可以有效克服光学成像系统景深有限性的媒体方式，正逐渐受到研究者的关注。然而，当前多聚焦视频的融合效果还不够理想，融合与编码过程也冗余复杂，没有有效利用多聚焦视频的原始码流信息，给多聚焦视频的应用普及带来了巨大的挑战。本项目将致力于多聚焦视频的融合与编码研究，以提升融合的质量与速度、编码的效率为目的，主要研究内容包括：针对现阶段还没有公开的具有标注信息的多聚焦视频数据集问题，研究合成数据集方式来构建大规模具有标注信息的训练数据集；利用深度神经网络良好的特征表征能力，构建适用于多聚焦视频融合的网络模型，并进一步研究如何优化融合过程。在融合层基础上，以利用多聚焦视频原始码流信息为基本思路，开展对融合视频的高效编码研究，为多聚焦视频应用奠定基础。最后，从定性估计及定量计算两个角度，研究如何评价融合质量，建立可靠的多聚焦融合视频质量评价方法，进一步指导融合及编码算法的性能提升。

Multi-focus video, as a media method that can effectively overcome the limited depth of field of the optical imaging system, is gradually attracting researchers' attention. However, the current fusion effect of multi-focus video is still not ideal, and the fusion and coding process is redundant and complex since the original bitstream information of multi-focus video is not effectively utilized, which bring great challenges to the application of multi-focus video. In this project, we focus on the fusion and coding of multi-focus video, with the aim of improving the quality and speed of fusion, and the efficiency of coding. First, since there are no public multi-focus video datasets with annotation information yet, we propose a synthesis method to build a large-scale training dataset with labeled information. By leveraging the well feature representation capabilities of deep neural networks, we establish a fusion model tailored to multi-focus video and further optimize the fusion process. On the basis of the fusion level, by exploiting the original bitstream information of multi-focus video, we try to achieve high-efficient coding of fusion video, laying the foundation for multi-focus video applications. Finally, from two perspectives of qualitative estimation and quantitative calculation, we establish a reliable multi-focus fusion video quality evaluation method, and further guide the performance improvement of fusion and coding algorithms.