Real-time object detection in 360-degree videos
Real-time object detection in 360-degree videos
复制标题
DOI：
10.1117/12.2586403
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Jounsup Park
中科院分区：
文献类型：
--
作者：
Jounsup Park
通过互联网流式传输360度视频是一项具有挑战性的任务，但它通过允许观众浏览360度内容提供了丰富的多媒体体验。与传统视频相比，360度视频在互联网上传输需要更大的带宽和更少的延迟。因此，必须从视频中丢弃不可见区域以节省带宽。视图预测技术已经被用于预测要流传输的360度视频帧的可见区域。使用观众的过去观看行为数据的线性回归对于预测观众的短期未来行为是有用的，当网络延迟长于预测范围时，这是无用的。对象检测技术有助于预测观看者的未来运动以获得更长的预测范围，因为观看者倾向于跟随吸引他们注意力的对象。然而，使用卷积神经网络的常规对象检测技术（诸如YOLO）难以应用于360度视频。当为了处理和存储目的而将球形360度视频投影成等矩形视频时，在360度视频中存在失真。相同的物体在等矩形视频中可以具有不同的形状，这取决于它们在球体中的角位置。因此，在本文中，我们提出了一种多方向投影（MDP）技术来检测360度视频中的对象。所提出的多方向投影技术减轻了等矩形视频中的失真，并将重定向的视频馈送到对象检测系统。因此，用传统视频数据集训练的神经网络可以在没有任何改变的情况下使用。实验结果表明，该方法有助于检测360度视频的边缘中的对象。
Streaming of 360-degree videos over the internet is challenging task, but it provides rich multimedia experiences by allowing viewers to navigate 360-degree contents. The 360-degree videos need larger bandwidth and less latency to be streamed over the internet than the conventional videos. Therefore, non-visible area must be discarded from the video to save bandwidth. View prediction techniques have been used to predict visible area of the 360-degree video frames to be streamed. Linear regression using viewer’s past viewing behavior data is useful to predict short-term future behavior of the viewer, which is not useful when the network delay is longer than the prediction horizon. Object detection techniques help predicting viewers’ future motion for longer prediction horizon since the viewers tend to follow the objects that draw their attention. However, conventional object detection techniques using a convolutional neural network, such as YOLO, are difficult to be applied to 360-degree videos. There are distortions in the 360-degree videos when the spherical 360-degree video is projected into equi-rectangular videos for processing and storing purposes. A same object could have different shapes in the equi-rectangular video depends on their angular position in the sphere. Therefore, in this paper, we propose a multi-directional projection (MDP) technique to detect objects in the 360-degree videos. The proposed multi- directional projection technique mitigates the distortions in the equi-rectangular videos and feeds the redirected videos to the object detection system. Therefore, the neural network trained with conventional video dataset can be used without any change. Experimental result shows that the proposed method helps detecting objects in the edges of the 360-degree videos.