Design of a Real-Time Human-Robot Collaboration System Using Dynamic Gestures
Design of a Real-Time Human-Robot Collaboration System Using Dynamic Gestures
复制标题
DOI：
10.1115/imece2020-23650
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Haodong Chen;M. Leu;Wenjin Tao;Zhaozheng Yin
中科院分区：
文献类型：
--
作者：
Haodong Chen;M. Leu;Wenjin Tao;Zhaozheng Yin
随着工业自动化和人工智能的发展，机器人系统正在发展成为工厂生产的重要组成部分，人机协作（HRC）成为工业领域的新趋势。在我们之前的工作中，我们设计了十种动态手势，用于制造场景中人类工人和机器人之间的通信，并开发了基于卷积神经网络（CNN）的动态手势识别模型。基于该模型，本研究旨在设计和开发一种基于多线程方法和CNN的实时HRC系统。该系统基于动态手势实现了工人与机械臂之间的实时交互。首先，构建多线程架构，实现高速运行和快速响应，同时调度多个任务；接下来，开发了一种实时动态手势识别算法，该算法连续监测和捕获人类工人的行为和运动，并实时生成运动历史图像（MHIs）。同时完成了mhi的生成和分类模型的识别。如果检测到指定的动态手势，它会立即传输到机械臂进行实时响应。为了实现手势识别的实时运动历史和分类结果的可视化，开发了一个集成该系统的图形用户界面（GUI）。通过与六自由度柯马工业机器人的实际协作实验，验证了所提系统的可行性和鲁棒性。
With the development of industrial automation and artificial intelligence, robotic systems are developing into an essential part of factory production, and the human-robot collaboration (HRC) becomes a new trend in the industrial field. In our previous work, ten dynamic gestures have been designed for communication between a human worker and a robot in manufacturing scenarios, and a dynamic gesture recognition model based on Convolutional Neural Networks (CNN) has been developed. Based on the model, this study aims to design and develop a new real-time HRC system based on multi-threading method and the CNN. This system enables the real-time interaction between a human worker and a robotic arm based on dynamic gestures. Firstly, a multi-threading architecture is constructed for high-speed operation and fast response while schedule more than one task at the same time. Next, A real-time dynamic gesture recognition algorithm is developed, where a human worker’s behavior and motion are continuously monitored and captured, and motion history images (MHIs) are generated in real-time. The generation of the MHIs and their identification using the classification model are synchronously accomplished. If a designated dynamic gesture is detected, it is immediately transmitted to the robotic arm to conduct a real-time response. A Graphic User Interface (GUI) for the integration of the proposed HRC system is developed for the visualization of the real-time motion history and classification results of the gesture identification. A series of actual collaboration experiments are carried out between a human worker and a six-degree-of-freedom (6 DOF) Comau industrial robot, and the experimental results show the feasibility and robustness of the proposed system.