Do End-to-end Stereo Algorithms Under-utilize Information?
Do End-to-end Stereo Algorithms Under-utilize Information?
复制标题
DOI：
10.1109/3dv50981.2020.00047
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Changjiang Cai;Philippos Mordohai
中科院分区：
文献类型：
--
作者：
Changjiang Cai;Philippos Mordohai
用于立体匹配的深度网络通常利用2D或3D卷积编码器-解码器架构来聚合成本并正则化成本量以进行准确的视差估计。由于内容不敏感的卷积以及下采样和上采样操作，这些成本聚合机制没有充分利用图像中可用的信息。视差图在遮挡边界附近过度平滑，以及在薄结构中的错误预测。在本文中，我们展示了如何将深度自适应滤波和可微半全局聚合集成到现有的2D和3D卷积网络中，以实现端到端立体匹配，从而提高精度。这些改进是由于利用来自图像的RGB信息作为信号来动态地指导匹配过程，除了作为我们试图在图像之间进行匹配的信号之外。我们在KITTI 2015和Virtual KITTI 2数据集上展示了广泛的实验结果，比较了四种立体网络（DispNetC，GCNet，PSMNet和GANet），将四种自适应滤波器（分割感知双边滤波，动态滤波网络，像素自适应卷积和半全局聚合）集成到其架构中。我们的代码可在https://github.com/ccj5351/DAFStereoNets上获得。
Deep networks for stereo matching typically leverage 2D or 3D convolutional encoder-decoder architectures to aggregate cost and regularize the cost volume for accurate disparity estimation. Due to content-insensitive convolutions and down-sampling and up-sampling operations, these cost aggregation mechanisms do not take full advantage of the information available in the images. Disparity maps suffer from over-smoothing near occlusion boundaries, and erroneous predictions in thin structures. In this paper, we show how deep adaptive filtering and differentiable semi-global aggregation can be integrated in existing 2D and 3D convolutional networks for end-to-end stereo matching, leading to improved accuracy. The improvements are due to utilizing RGB information from the images as a signal to dynamically guide the matching process, in addition to being the signal we attempt to match across the images. We show extensive experimental results on the KITTI 2015 and Virtual KITTI 2 datasets comparing four stereo networks (DispNetC, GCNet, PSMNet and GANet) after integrating four adaptive filters (segmentation-aware bilateral filtering, dynamic filtering networks, pixel adaptive convolution and semi-global aggregation) into their architectures. Our code is available at https://github.com/ccj5351/DAFStereoNets.