A Supervised Approach to Predicting Noise in Depth Images
A Supervised Approach to Predicting Noise in Depth Images
复制标题
预测深度图像中噪声的监督方法
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Russ Tedrake
中科院分区：
文献类型：
--
作者：
Chris Sweeney;Gregory Izatt;Russ Tedrake
现代机器人系统非常复杂，需要在模拟中使用详细的传感器噪声模型进行测试，以有效验证机器人行为。深度图像特别是带有明显的噪声，其形式为场景相关的像素丢失和失真。不幸的是，许多深度相机模拟包含有限的噪声模型，或者只能支持生成简单场景的真实深度图像，这限制了它们在有效测试感知算法方面的有用性。我们提出了一种数据驱动的方法，通过使用卷积神经网络（CNN）来预测模拟无噪声深度图像的哪些像素不会返回（无深度返回像素，或NDP），为复杂的模拟环境生成更逼真的噪声。我们选择在这里关注NDP，因为这些丢失是深度图像噪声的最常见和最引人注目的形式。为了训练这个网络，我们使用从标签融合数据集重建的真实世界场景，为用于扫描场景的每个噪声深度图像提供地面真实深度。我们使用产生的无噪声和有噪声的深度图像对作为标记的示例，并训练网络来预测无噪声图像的哪些像素将是NDP。当用于对深度传感器的模拟进行后处理时，即使在混乱的场景中，该系统也会产生逼真的深度图像。为了证明我们的方法成功地缩小了深度图像的现实差距，我们证明了用于对象姿态估计的流行ICP算法在我们的CNN损坏的模拟深度图像上比在未损坏的深度图像和无监督的域自适应基线上更真实地失败。
Modern robotic systems are very complex and need to be tested in simulations with detailed sensor noise models to effectively verify robotic behavior. Depth imagery in particular comes with significant noise in the form of scene-dependent pixel-wise dropouts and distortions. Unfortunately, many depth camera simulations contain limited noise models, or can only support generating realistic depth images of simple scenes, which limits their usefulness in effectively testing perception algorithms. We propose a data driven approach to generate more realistic noise for complex simulated environments by using a convolutional neural network (CNN) to predict which pixels of a simulated noise-free depth image will not have returns (no-depth-return pixels, or NDP). We choose to focus on NDP here, as these dropouts are the most common and dramatic form of depth image noise. To train this network, we use reconstructed real-world scenes from the Label Fusion dataset to provide ground truth depth for each noisy depth image used to scan the scene. We use the resulting noise-free and noisy depth image pairs as labeled examples and train the network to predict which pixels of the noise-free image will be NDP. When used to post-process a simulation of a depth sensor, this system produces realistic depth images, even in cluttered scenes. To demonstrate that our approach successfully closes the reality gap for depth imagery, we show that the popular ICP algorithm for object pose estimation fails more realistically on our CNN-corrupted simulated depth images than on uncorrupted depth images and unsupervised domain adaptation baselines.