Fast Depth Prediction and Obstacle Avoidance on a Monocular Drone Using Probabilistic Convolutional Neural Network
Fast Depth Prediction and Obstacle Avoidance on a Monocular Drone Using Probabilistic Convolutional Neural Network
复制标题
使用概率卷积神经网络对单目无人机进行快速深度预测和避障
DOI：
10.1109/tits.2019.2955598
复制
发表时间：
2019-12
影响因子：
8.5
通讯作者：
Cheng Kwang-Ting
中科院分区：
文献类型：
--
作者：
Yang Xin;Chen Jingyu;Dang Yuanjie;Luo Hongcheng;Tang Yuesheng;Liao Chunyuan;Chen Peng;Cheng Kwang-Ting
最近的研究采用先进的深度卷积神经网络（CNN）进行单目深度感知，这在依赖低/中级GPU（例如TX 2和1050Ti）进行计算的小型无人机上几乎无法有效运行。此外，可以有效地和高效地产生具有模型置信度的概率深度预测的方法还没有得到很好的研究。缺乏这样的方法可能会在无人机应用中产生错误的、有时是致命的决策（例如，在深度大但估计置信度低的区域中选择航路点）。本文提出了一种使用轻量级概率CNN（pCNN）进行单目深度预测和避障的实时机载方法，该方法非常适合用于轻型节能无人机。对于每个视频帧，我们的pCNN可以有效地预测其深度图和相应的置信度。我们的轻量级pCNN的准确性通过将来自视觉里程计的稀疏深度估计集成到网络中来指导密集深度和置信度推断而大大提高。通过将无人机的动态运动约束和置信度值两者嵌入到空间深度图中，将所估计的深度图变换到自我动态空间（EDS）中。EDS中自动计算可穿越的航路点，并根据这些航路点为无人机产生适当的控制输入。在公开数据集上的大量实验结果表明，我们的深度预测方法在TX 2和1050Ti GPU上分别运行在12 Hz和45 Hz，比现有方法快1.8倍~ 5.6倍，并获得更好的深度估计精度。我们还在模拟和真实的环境中进行了避障实验，以证明我们的方法比基线方法的优越性。
Recent studies employ advanced deep convolutional neural networks (CNNs) for monocular depth perception, which can hardly run efficiently on small drones that rely on low/middle-grade GPU(e.g. TX2 and 1050Ti) for computation. In addition, the methods which can effectively and efficiently produce probabilistic depth prediction with a measure of model confidence have not been well studied. The lack of such a method could yield erroneous, sometimes fatal, decisions in drone applications (e.g. selecting a waypoint in a region with a large depth yet a low estimation confidence). This paper presents a real-time onboard approach for monocular depth prediction and obstacle avoidance with a lightweight probabilistic CNN (pCNN), which will be ideal for use in a lightweight energy-efficient drone. For each video frame, our pCNN can efficiently predict its depth map and the corresponding confidence. The accuracy of our lightweight pCNN is greatly boosted by integrating sparse depth estimation from a visual odometry into the network for guiding dense depth and confidence inference. The estimated depth map is transformed into Ego Dynamic Space (EDS) by embedding both dynamic motion constraints of a drone and the confidence values into the spatial depth map. Traversable waypoints are automatically computed in EDS based on which appropriate control inputs for the drone are produced. Extensive experimental results on public datasets demonstrate that our depth prediction method runs at 12Hz and 45Hz on TX2 and 1050Ti GPU respectively, which is 1.8X~5.6X faster than the state-of-the-art methods and achieves better depth estimation accuracy. We also conducted experiments of obstacle avoidance in both simulated and real environments to demonstrate the superiority of our method to the baseline methods.
登录
查看更多内容
DOI：
--
发表时间：
2003
期刊：
--
影响因子：
--
作者：
Richard Hartley;Andrew Zisserman
通讯作者：
Richard Hartley;Andrew Zisserman
DOI：
10.1017/cbo9780511811685.009
发表时间：
2001-04
期刊：
--
影响因子：
--
作者：
Richard Hartley;Andrew Zisserman
通讯作者：
Richard Hartley;Andrew Zisserman
DOI：
10.1109/iros.2012.6386146
发表时间：
2012-12
期刊：
2012 IEEE/RSJ International Conference on Intelligent Robots and Systems
影响因子：
--
作者：
Ian Lenz;Mevlana Gemici;Ashutosh Saxena
通讯作者：
Ian Lenz;Mevlana Gemici;Ashutosh Saxena
DOI：
--
发表时间：
2017-09
期刊：
--
影响因子：
--
作者：
Huan Wang;Qiming Zhang;Yuehai Wang;Haoji Hu
通讯作者：
Huan Wang;Qiming Zhang;Yuehai Wang;Haoji Hu
DOI：
10.1109/irds.2002.1041455
发表时间：
2002-12
期刊：
IEEE/RSJ International Conference on Intelligent Robots and Systems
影响因子：
--
作者：
J. Minguez;L. Montano;O. Khatib
通讯作者：
J. Minguez;L. Montano;O. Khatib