FESTA: Flow Estimation via Spatial-Temporal Attention for Scene Point Clouds
FESTA: Flow Estimation via Spatial-Temporal Attention for Scene Point Clouds
复制标题
DOI：
10.1109/cvpr46437.2021.01395
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Haiyan Wang;Jiahao Pang;M. Lodhi;Yingli Tian;Dong Tian
中科院分区：
文献类型：
--
作者：
Haiyan Wang;Jiahao Pang;M. Lodhi;Yingli Tian;Dong Tian
场景流描述了3D场景的动态，这对于自动驾驶、机器人导航、AR/VR等各种应用都是至关重要的。传统上，场景流是根据密集/规则的RGB视频帧来估计的。随着深度传感技术的发展，通过点云进行精确的三维测量成为可能，这引发了对三维场景流动的新研究。然而，由于典型点云采样模式的稀疏性和不规则性，从点云中提取场景流仍然具有挑战性。与不规则采样相关的一个主要问题是点集提取/特征提取过程中的随机性，这是许多流量估计场景中的基本过程。针对这种不稳定的抽象问题，提出了一种新的带注意力的空间抽象(SA2)层。此外，还提出了一种带注意力的时间抽象(TA2)层，用于在时间域纠正注意，从而在更大范围内扩大运动范围。大量的分析和实验验证了我们的方法的动机和显著的性能提高，被称为基于时空注意力的流估计(FETA)，与几种最先进的场景流估计基准相比较。
Scene flow depicts the dynamics of a 3D scene, which is critical for various applications such as autonomous driving, robot navigation, AR/VR, etc. Conventionally, scene flow is estimated from dense/regular RGB video frames. With the development of depth-sensing technologies, precise 3D measurements are available via point clouds which have sparked new research in 3D scene flow. Nevertheless, it remains challenging to extract scene flow from point clouds due to the sparsity and irregularity in typical point cloud sampling patterns. One major issue related to irregular sampling is identified as the randomness during point set abstraction/feature extraction—an elementary process in many flow estimation scenarios. A novel Spatial Abstraction with Attention (SA2) layer is accordingly proposed to alleviate the unstable abstraction problem. Moreover, a Temporal Abstraction with Attention (TA2) layer is proposed to rectify attention in temporal domain, leading to benefits with motions scaled in a larger range. Extensive analysis and experiments verified the motivation and significant performance gains of our method, dubbed as Flow Estimation via Spatial-Temporal Attention (FESTA), when compared to several state-of-the-art benchmarks of scene flow estimation.