Joint Detection of Motion Boundaries and Occlusions
Joint Detection of Motion Boundaries and Occlusions
复制标题
DOI：
--
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
Hannah Kim;Shuzhi Yu;Carlo Tomasi
中科院分区：
文献类型：
--
作者：
Hannah Kim;Shuzhi Yu;Carlo Tomasi
我们提出了MONET，这是一种卷积神经网络，它在时间上向前和向后联合检测视频中的运动边界(MBS)和遮挡区域(OCC)。检测是困难的，因为光流沿着MBS是不连续的，并且在OCCS中没有定义，而许多流估计器假设平滑并且到处定义流。为了同时在两个时间方向上进行推理，我们在两个帧之间直接扭曲估计的地图。由于帧之间的外观不匹配通常表示MBS或OCC的附近，我们构造了一个代价块，该代价块为一帧中的每个特征记录与搜索范围内匹配特征的最小差异。这种成本块是二维的，比流动分析中使用的四维成本块便宜得多。成本块特征由编码器计算，MB和OCC估计由解码器计算。我们发现，将解码器层按精细到粗略排列，而不是按粗略到精细排列，可以提高性能。在没有任何微调的情况下，Monet在Sintel和FlyingChairsOCC基准测试上的表现都超过了以前的最先进水平。
We propose MONet, a convolutional neural network that jointly detects motion boundaries (MBs) and occlusion regions (Occs) in video both forward and backward in time. Detection is difficult because optical flow is discontinuous along MBs and undefined in Occs, while many flow estimators assume smoothness and a flow defined everywhere. To reason in the two time directions simultaneously, we direct-warp the estimated maps between the two frames. Since appearance mismatches between frames often signal vicinity to MBs or Occs, we construct a cost block that for each feature in one frame records the lowest discrepancy with matching features in a search range. This cost block is two-dimensional, and much less expensive than the four-dimensional cost volumes used in flow analysis. Cost-block features are computed by an encoder, and MB and Occ estimates are computed by a decoder. We found that arranging decoder layers fine-to-coarse, rather than coarse-to-fine, improves performance. MONet outperforms the prior state of the art for both tasks on the Sintel and FlyingChairsOcc benchmarks without any fine-tuning on them.