Fast and High-Quality Monocular Depth Estimation with Optical Flow for Autonomous Drones
Fast and High-Quality Monocular Depth Estimation with Optical Flow for Autonomous Drones
复制标题
DOI：
10.3390/drones7020134
复制
发表时间：
2023-02
期刊：
影响因子：
4.8
通讯作者：
Tomoyasu Shimada;Hiroki Nishikawa;Xiangbo Kong;Hiroyuki Tomiyama
中科院分区：
文献类型：
--
作者：
Tomoyasu Shimada;Hiroki Nishikawa;Xiangbo Kong;Hiroyuki Tomiyama
近年来，自主无人机因其便利性在许多领域引起了人们的关注。自主无人机需要精确的深度信息以避免碰撞才能快速飞行，在基于卷积神经网络(CNN)的应用中，经常使用RGB图像和LiDAR点云来估计到障碍物的距离。这样的应用是在嵌入式系统上实现的。为了准确地估计深度，这类CNN模型一般都很复杂，需要提取许多特征，从而增加了计算复杂度，需要很长的推理时间。为了解决这一问题，我们使用光流来辅助深度估计。此外，我们提出了一种新的关注结构，该结构最大限度地利用了光流，而不会使网络复杂化。此外，通过在训练中加入感知鉴别器，在不修改深度估计器的情况下，我们获得了更好的性能。在Kitti数据集上通过精度、误差和推理时间对所提出的模型进行了评估。实验表明，与以往的方法相比，该方法在Jetson Nano上获得了高达34%的准确率、55%的错误率和66%的推理时间。通过模拟无人机飞行中的避碰实验对该方法进行了评估，获得了所有估计方法中最低的碰撞率。这些实验结果表明了所提出的方法在现实世界中应用于自主无人机飞行的潜力。
Recent years, autonomous drones have attracted attention in many fields due to their convenience. Autonomous drones require precise depth information so as to avoid collision to fly fast and both of RGB image and LiDAR point cloud are often employed in applications based on Convolutional Neural Networks (CNNs) to estimate the distance to obstacles. Such applications are implemented onboard embedded systems. In order to precisely estimate the depth, such CNN models are in general so complex to extract many features that the computational complexity increases, requiring long inference time. In order to solve the issue, we employ optical flow to aid in-depth estimation. In addition, we propose a new attention structure that makes maximum use of optical flow without complicating the network. Furthermore, we achieve improved performance without modifying the depth estimator by adding a perceptual discriminator in training. The proposed model is evaluated through accuracy, error, and inference time on the KITTI dataset. In the experiments, we have demonstrated the proposed method achieves better performance by up to 34% accuracy, 55% error reduction and 66% faster inference time on Jetson nano compared to previous methods. The proposed method is also evaluated through a collision avoidance in simulated drone flight and achieves the lowest collision rate of all estimation methods. These experimental results show the potential of proposed method to be used in real-world autonomous drone flight applications.