MASD: A Multimodal Assembly Skill Decoding System for Robot Programming by Demonstration
MASD: A Multimodal Assembly Skill Decoding System for Robot Programming by Demonstration
复制标题
MASD：用于机器人编程演示的多模态装配技能解码系统
DOI：
10.1109/tase.2017.2783342
复制
发表时间：
2018-01
影响因子：
5.6
通讯作者：
Yong Liu
中科院分区：
文献类型：
--
作者：
Yue Wang;Yanmei Jiao;Rong Xiong;Hongsheng Yu;Jiafan Zhang;Yong Liu
演示编程（PBD）将机器人编程从代码级转变为机器人与人类之间的自动化接口，提升了机器人自动化的灵活性。在本文中，我们重点关注对工业机器人进行装配任务的编程，将人类演示解析为一系列装配技能并将该技能编译为机器人可执行文件。为了实现这一目标，提出了一种使用多模态信息来识别装配技能的识别系统，称为MASD，包括：1）初始学习阶段，使用分层模型通过考虑动作-物体效果、手势和轨迹的特征来识别动作；2）回顾性思维阶段，使用分割方法将连续演示最佳地切割为多个装配技能。使用 MASD，可以实时高精度地解释装配任务的演示，从而推动了这样一个假设：MASD 顶部的 PBD 系统可以扩展到更现实的装配任务，而不仅仅是纯粹的位置移动和拾取。在实验中，技能识别模块在单个和多个组装技能演示中均用于识别五种组装技能，并且优于比较动作识别方法。除了与MASD集成外，PBD系统还可以根据演示生成程序，并成功使ABB工业机械臂模拟器组装手电筒和开关，验证了最初的假设。从业者须知——在传统的机器人自动化中，机器人的关键作用主要在于其能够长期高速、准确地重复各种任务，而部署的编程成本则需要数天到数月的时间。另一方面，定制化的新趋势带来了新的特点：生产周期短、批量小。这种不可逆转的动力促使机器人有效地从一个任务切换到另一个任务。这里最大的瓶颈就是繁琐的编程，这对大多数制造业从业者来说也有很高的前提条件。这种情况促进了 PBD 系统的开发，该系统可以理解人类专家在演示中执行的组装技能，并相应地生成机器人执行教学任务的程序。在本文中，我们提出了一种技能解码系统，将观察到的原始演示解析为符号序列，这是实现自动编程的关键桥梁。该系统综合考虑了装配背景中的环境可控、计算资源有限等优点和缺点，实现了高性能的识别，并针对装配任务中的PBD进行了量身定制。它对于基于一组标准零件的模块化操作的装配任务特别有用。从工业应用的角度来看，基于所提出系统的PBD是提高制造灵活性的一种有前途的解决方案，预计这将在中期实现，但这是朝着这一目标迈出的重要一步。
Programming by demonstration (PBD) transforms the robot programming from the code level to automated interface between robot and human, promoting the flexibility of robotized automation. In this paper, we focus on programming the industrial robot for assembly tasks by parsing the human demonstration into a series of assembly skills and compiling the skill to the robot executables. To achieve this goal, an identification system using multimodal information to recognize the assembly skill, called MASD, is proposed including: 1) an initial learning stage using a hierarchical model to recognize the action by considering the features from action-object effect, gesture, and trajectory and 2) a retrospective thinking stage using a segmentation method to cut the continuous demonstrations into multiple assembly skills optimally. Using MASD, the demonstration of assembly tasks can be explained with high accuracy in real time, driving a hypothesis that a PBD system on the top of MASD can be extended to more realistic assembly tasks beyond pure positional moving and picking. In experiments, the skill identification module is used to recognize the five kinds of assembly skills in demonstrations of both single and multiple assembly skills, and outperforms the comparative action identification methods. Besides integrated with the MASD, the PBD system can generate the program based on the demonstration and successfully enable an ABB industrial robotic arm simulator to assemble a flashlight and a switch, verifying the initial hypothesis. Note to Practitioners—In the conventional robotized automation, the key role of the robot mainly owes to its capacity for repeating a wide variety of tasks with high speed and accuracy in long term, with a cost of days to months of programming for deployment. On the other hand, the new trend of customization brings the new characteristics: production in short cycle and small volume. This irreversible momentum urges the robot to switch from task to task efficiently. The biggest bottleneck here is the tedious programming, which also has high prerequisites for most practitioners in manufacturing. This situation motivates the development of a PBD system that can understand the assembly skills performed by the human experts in the demonstration and accordingly generate the program for robot’s execution of the taught task. In this paper, we present a skill decoding system to parse the observational raw demonstration into symbolic sequences, which is the crucial bridge to enable the automatic programming. The system achieves high performance in recognition and is tailored for the PBD in assembly tasks by considering both advantages and disadvantages in the background of assembly, such as controllable environment and limited computational resources. It is particularly useful for assembly tasks with modularized actions based on a set of standard parts. At the perspective of industrial application, the PBD upon the proposed system is a promising solution to improve the flexibility of manufacture, which is expected to be true in midterm but an important step toward this goal.
登录
查看更多内容
DOI：
10.1109/icpr.2008.4761080
发表时间：
2008-12
期刊：
2008 19th International Conference on Pattern Recognition
影响因子：
--
作者：
M. Elmezain;A. Al-Hamadi;Jörg Appenrodt;B. Michaelis
通讯作者：
M. Elmezain;A. Al-Hamadi;Jörg Appenrodt;B. Michaelis
DOI：
10.1109/robot.2010.5509319
发表时间：
2010-05
期刊：
2010 IEEE International Conference on Robotics and Automation
影响因子：
--
作者：
E. Aksoy;A. Abramov;F. Wörgötter;B. Dellen
通讯作者：
E. Aksoy;A. Abramov;F. Wörgötter;B. Dellen
影响因子：
19.5
作者：
Laptev, I
通讯作者：
Laptev, I
影响因子：
0.8
作者：
Mohammad Abdul Rahman;Mohammad Kabir Ahmed;Mohammad Khir Harun;Mohd Harridon Mohamed Suffian;Mohd Izhar Abu Bakar-Mohd
通讯作者：
Mohammad Abdul Rahman;Mohammad Kabir Ahmed;Mohammad Khir Harun;Mohd Harridon Mohamed Suffian;Mohd Izhar Abu Bakar-Mohd
DOI：
10.1109/cyber.2014.6917456
发表时间：
2014-10
期刊：
The 4th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent
影响因子：
--
作者：
Yue Wang;R. Xiong;Longbin Shen;K. Sun;Jiafan Zhang;Liwei Qi
通讯作者：
Yue Wang;R. Xiong;Longbin Shen;K. Sun;Jiafan Zhang;Liwei Qi