Real-time Object Detection with Deep Learning for Robot Vision on Mixed Reality Device
Real-time Object Detection with Deep Learning for Robot Vision on Mixed Reality Device
复制标题
DOI：
10.1109/lifetech52111.2021.9391811
复制
发表时间：
2021-03
期刊：
影响因子：
--
通讯作者：
Jiazhen Guo;Peng Chen;Yinlai Jiang;H. Yokoi;Shunta Togo
中科院分区：
文献类型：
--
作者：
Jiazhen Guo;Peng Chen;Yinlai Jiang;H. Yokoi;Shunta Togo
混合现实设备感测能力对机器人来说很有价值，例如，惯性测量单元（IMU）传感器和飞行时间（TOF）深度传感器可以支持机器人导航其环境。本文展示了一个在混合现实设备上实现的深度学习（YOLO模型）背景、实时对象检测系统。该系统的目标是在HoloLens和Ubuntu系统之间创建一个实时通信系统，以便使用YOLO模型进行实时对象检测。实验结果表明，该方法具有较快的速度，能够实现HoloLens的实时目标检测。这使得Microsoft HoloLens成为机器人视觉设备。为了增强人机交互，我们将在未来将其应用于可穿戴机器人手臂系统，以自动抓取物体。
Mixed reality device sensing capabilities are valuable for robots, for example, the inertial measurement unit (IMU) sensor and time-of-flight (TOF) depth sensor can support the robot in navigating its environment. This paper demonstrates a deep learning (YOLO model) background, realtime object detection system implemented on mixed reality device. The goal of the system is to create a real-time communication system between HoloLens and Ubuntu systems to enable real-time object detection using the YOLO model. The experimental results show that the proposed method has a fast speed to achieve real-time object detection using HoloLens. This enables Microsoft HoloLens as a device for robot vision. To enhance human-robot interaction, we will apply it to a wearable robot arm system to automatically grasp objects in the future.