Learning-based Homography Matrix Optimization for Dual-fisheye Video Stitching
Learning-based Homography Matrix Optimization for Dual-fisheye Video Stitching
复制标题
DOI：
10.1145/3609395.3610600
复制
发表时间：
2023-09
期刊：
影响因子：
--
通讯作者：
M. Zhu;Yang Sui;Bo Yuan;Yao-Chang Liu
中科院分区：
文献类型：
--
作者：
M. Zhu;Yang Sui;Bo Yuan;Yao-Chang Liu
在本文中，我们提出了一种新的基于特征的视频拼接算法拼接背靠背鱼眼摄像机视频成一个全向视频在视频直播场景。我们的主要贡献在于基于学习的方法，通过梯度下降在线细化单应性矩阵。通过在特征点的滚动数据集上进行训练来更新单应性矩阵，所述特征点在捕获新视频帧时被提取和匹配。实验结果表明，我们的方法可以创建拼接图像，更好地对齐匹配的功能与更低的均方误差（MSE）比传统的基于特征的拼接方法。此外，与使用基于校准的拼接的供应商提供的软件（VUZE VR Studio）相比，我们的方法也产生了明显更好的结果。
In this paper, we propose a novel feature-based video stitching algorithm for stitching back-to-back fisheye camera videos into one omnidirectional video in a video live streaming scenario. Our main contribution lies in a learning-based approach that refines the homography matrix in an online manner via gradient descent. The homography matrix is updated by training on a rolling dataset of feature points that are extracted and matched as new video frames are captured. Experimental results show that our method can create stitched images that better align matching features with lower mean squared error (MSE) than traditional feature-based stitching method. Furthermore, compared to vendor-supplied software (VUZE VR Studio) that uses calibration-based stitching, our method also produces visibly better results.