NRI: FND: Intelligent Co-robots for Complex Welding Manufacturing through Learning and Generalization of Welders Capabilities

NRI：FND：通过学习和推广焊工能力实现复杂焊接制造的智能协作机器人

• 批准号: 2024614
• 负责人: YuMing Zhang
• 金额: $ 66.55万
• 依托单位: University of Kentucky Research Foundation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2024614&HistoricalAwards=false
• 关键词: NRI; FND; Intelligent; Co; robots

There is a dramatic and growing shortage of highly skilled welders, accentuated by the fact that manufacturing complexity and production volumes are rising. As a result, global use of robotic welding is expanding rapidly. However, current welding robots are not as adaptive and creative as human welders in performing complex welding tasks that require sophisticated skills. This award supports fundamental research on advancing the robotic capabilities needed to realize fully robotic automation of complex welding tasks. The research will endow collaborative welding robots with sophisticated welding knowledge, expert intelligence, and an interactive learning capability to enable them to address dynamic welding scenarios. The research results will both enhance the scientific base for robotic control and facilitate the realization of fully automatic, robotic, and intelligent manufacturing. The research involves several disciplines, including welding, process monitoring, data visualization, machine learning, optimization, and robotic control. That multi-disciplinary approach will broaden the participation of students from diverse backgrounds in research, and the knowledge gained will be incorporated in curricula in robotic and intelligent manufacturing. The double-electrode, gas metal arc welding process is complex, requiring intense collaboration between expert welders. As a result, the robotic automation of such a complex welding process requires advances in the scientific base of robotic perception, learning, and control. The project will research advanced methods for the extraction of expert welding-domain knowledge and the quantification and interpretation of that knowledge for use by collaborative robots, thereby equipping collaborative welding robots to perform complex welding tasks. To realize that goal, the research team will: 1) build an immersive virtual reality system with a three-dimensional rendering of the weld pool and arc that can characterize the weld scene and record human operations, 2) use an explainable recurrent convolutional neural network to perform causal analysis of the torch manipulation of human welders to obtain its relationship to dynamic weld pool/arc evolution, 3) generalize the results in terms of human heterogeneity by using transfer learning to extract common latent knowledge from different human welders, and 4) develop an interactive learning module that allows collaborative robots to be supervised by on-site human welders through the reinforcement learning-based perception of language instructions and human gestures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

高技能焊工的短缺日益严重，制造复杂性和产量不断上升的事实加剧了这一问题。因此，全球机器人焊接的使用正在迅速扩大。 然而，当前的焊接机器人在执行需要复杂技能的复杂焊接任务时不像人类焊工那样具有适应性和创造性。该奖项支持基础研究，以提高实现复杂焊接任务的完全机器人自动化所需的机器人能力。该研究将赋予协作焊接机器人复杂的焊接知识，专家智能和交互式学习能力，使他们能够解决动态焊接场景。研究成果将为机器人控制奠定科学基础，促进全自动、机器人和智能制造的实现。该研究涉及多个学科，包括焊接，过程监控，数据可视化，机器学习，优化和机器人控制。这种多学科方法将扩大来自不同背景的学生对研究的参与，所获得的知识将纳入机器人和智能制造课程。 双电极气体保护金属极电弧焊工艺复杂，需要专业焊工之间的密切合作。 因此，这种复杂焊接过程的机器人自动化需要机器人感知、学习和控制的科学基础的进步。该项目将研究先进的方法，用于提取专家焊接领域知识，并量化和解释该知识，供协作机器人使用，从而使协作焊接机器人能够执行复杂的焊接任务。为了实现这一目标，研究小组将：1）构建具有焊接熔池和电弧的三维渲染的沉浸式虚拟现实系统，其可以表征焊接场景并记录人类操作，2）使用可解释的递归卷积神经网络来执行人类焊工的焊炬操纵的因果分析，以获得其与动态焊接熔池/电弧演变的关系，3）通过使用迁移学习来从不同的人类焊工提取共同的潜在知识，以及4）开发交互式学习模块，其允许协作机器人通过强化学习由现场人类焊工监督-该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Data-driven process characterization and adaptive control in robotic arc welding

• DOI: 10.1016/j.cirp.2022.04.046
• 发表时间: 2022-05
• 期刊: CIRP Annals
• 作者: Peng Wang;J. Kershaw;Matthew Russell;Jianjing Zhang;Yuming Zhang;R. X. Gao

Do We Need a New Foundation to Use Deep Learning to Monitor Weld Penetration?

• DOI: 10.1109/lra.2023.3270038
• 发表时间: 2023-06
• 期刊: IEEE Robotics and Automation Letters
• 影响因子: 5.2
• 作者: Edison Mucllari;Rui Yu;Yue Cao;Qiang Ye;Yuming Zhang

Hybrid machine learning-enabled adaptive welding speed control

• DOI: 10.1016/j.jmapro.2021.09.023
• 发表时间: 2021-11
• 期刊: Journal of Manufacturing Processes
• 影响因子: 6.2
• 作者: J. Kershaw;Rui Yu;Yuming Zhang;Peng Wang

How to Accurately Monitor the Weld Penetration From Dynamic Weld Pool Serial Images Using CNN-LSTM Deep Learning Model?

• DOI: 10.1109/lra.2022.3173659
• 发表时间: 2022-07
• 期刊: IEEE Robotics and Automation Letters
• 影响因子: 5.2
• 作者: Rui Yu;J. Kershaw;Peng Wang;Yuming Zhang

Monitoring of Backside Weld Bead Width from High Dynamic Range Images Using CNN Network

• DOI: 10.1109/codit55151.2022.9804041
• 发表时间: 2022-05
• 期刊: 2022 8th International Conference on Control, Decision and Information Technologies (CoDIT)
• 作者: Rui Yu;J. Kershaw;Peng Wang;Yuming Zhang