NRI: INT: COLLAB: Manufacturing USA: Intelligent Human-Robot Collaboration for Smart Factory

NRI：INT：COLLAB：美国制造：智能工厂的智能人机协作

• 批准号: 1954548
• 负责人: Zhaozheng Yin
• 金额: $ 66.06万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-31 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1954548&HistoricalAwards=false
• 关键词: NRI; INT; COLLAB; Manufacturing; USA

This National Robotics Initiative (NRI) collaborative research project addresses the NSF Big Idea of Work at the Human-Technology Frontier by targeting human-robot collaboration in manufacturing. Recent advances in sensing, computational intelligence, and big data analytics have been rapidly transforming and revolutionizing the manufacturing industry towards robot-rich and digitally connected factories. However, effective, efficient and safe coordination between humans and robots on the factory floor has remained a significant challenge. To meet the need for safe and effective human-robot collaboration in manufacturing, the investigators will research an integrated set of algorithms and robotic test beds to sense, understand, predict and control the interaction of human workers and robots in collaborative manufacturing cells. It is expected that these methods will significantly improve the safety and productivity of hybrid human-robot production systems, thereby promoting their deployment in future "smart factories". To broaden the impact of this project, a partnership with Manufacturing USA Institute(s) and professional societies will be established to provide human-robot collaboration learning modules for inclusion in robotics and smart manufacturing-related curricula. These learning modules, together with annual events aimed at community college and pre-college students, and workshops for the dissemination of research results will raise public awareness and attract new entrants into the manufacturing and robotics industries, creating truly synergetic education opportunities in science, technology, engineering and mathematics, as well as accelerating the adoption of smart factory-enabling technologies. The project will address fundamental challenges in human-robot collaboration in the manufacturing environment, such as the limitation of one-to-one sensing between humans and robots, the lack of adaptive and stochastic modeling methods for reliable recognition and prediction of human actions and motions in different manufacturing scenarios, and multi-scale human-robot coordination. To address these challenges, multi-disciplinary research involving sensing, machine learning, stochastic modeling, robot path planning, and advanced manufacturing will be performed. Specific tasks include algorithm development and deployment on lab-scale and real-world test beds to: (1) sense and recognize where objects (e.g., robots, humans, parts or tools) are located and what each worker is doing; (2) predict what the next human action will be; and (3) plan and control safe and optimal robot trajectories for individualized on-the-job assistance for humans, proactively avoiding worker injury. The outcomes from the project will be evaluated on the shop-floor at the collaborating company COsorizio MAcchine Uensili (COMAU) in Michigan, and the Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing of the National Research Council of Italy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这个国家机器人计划（NRI）合作研究项目通过针对制造业中的人机协作，解决了NSF在人类技术前沿的工作理念。 传感、计算智能和大数据分析方面的最新进展正在迅速改变和彻底改变制造业，使其转向机器人丰富和数字连接的工厂。然而，在工厂车间，人类和机器人之间的有效、高效和安全协调仍然是一个重大挑战。为了满足制造业中安全有效的人机协作需求，研究人员将研究一套集成的算法和机器人测试床，以感知、理解、预测和控制协同制造单元中人类工人和机器人的交互。 预计这些方法将显著提高人机混合生产系统的安全性和生产率，从而促进其在未来“智能工厂”中的部署。 为了扩大该项目的影响，将与美国制造研究所和专业协会建立合作伙伴关系，提供人机协作学习模块，以纳入机器人和智能制造相关课程。 这些学习模块，以及针对社区大学和大学预科学生的年度活动，以及传播研究成果的研讨会，将提高公众意识，吸引新进入制造业和机器人行业的人，在科学，技术，工程和数学方面创造真正协同的教育机会，并加速采用智能工厂使能技术。 该项目将解决制造环境中人机协作的基本挑战，例如人类和机器人之间一对一传感的限制，缺乏自适应和随机建模方法来可靠地识别和预测不同制造场景中的人类动作和运动，以及多尺度人机协调。为了应对这些挑战，将进行涉及传感，机器学习，随机建模，机器人路径规划和先进制造的多学科研究。 具体任务包括在实验室规模和真实世界测试床上的算法开发和部署，以：（1）感测和识别物体（例如，机器人、人、零件或工具）的位置以及每个工人正在做什么;（2）预测下一个人类动作将是什么;以及（3）计划和控制安全和最佳的机器人轨迹，用于为人类提供个性化的工作辅助，主动避免工人受伤。该项目的成果将在密歇根州的合作公司COsorizio MAcchine Uensili（COMAU）和意大利国家研究理事会先进制造智能工业技术和系统研究所的车间进行评估。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

项目成果

Action Completeness Modeling with Background Aware Networks for Weakly-Supervised Temporal Action Localization

• DOI: 10.1145/3394171.3413687
• 发表时间: 2020-10
• 期刊: Proceedings of the 28th ACM International Conference on Multimedia
• 作者: M. Moniruzzaman;Zhaozheng Yin;Zhihai He;Ruwen Qin;M. Leu

Dynamic Gesture Design and Recognition for Human-Robot Collaboration With Convolutional Neural Networks

• DOI: 10.1115/isfa2020-9609
• 发表时间: 2020-07
• 期刊: 2020 International Symposium on Flexible Automation
• 作者: Haodong Chen;Wenjin Tao;M. Leu;Zhaozheng Yin

Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing

• DOI: 10.1016/j.jmsy.2020.02.010
• 发表时间: 2020-04-01
• 期刊: JOURNAL OF MANUFACTURING SYSTEMS
• 影响因子: 12.1
• 作者: Lai, Ze-Hao;Tao, Wenjin;Yin, Zhaozheng
• 通讯作者: Yin, Zhaozheng

Fusing and refining convolutional neural network models for assembly action recognition in smart manufacturing

• DOI: 10.1177/0954406220931547
• 发表时间: 2020-06-11
• 期刊: PROCEEDINGS OF THE INSTITUTION OF MECHANICAL ENGINEERS PART C-JOURNAL OF MECHANICAL ENGINEERING SCIENCE
• 影响因子: 2
• 作者: Al-Amin, Md.;Qin, Ruwen;Leu, Ming C.
• 通讯作者: Leu, Ming C.

Class-Aware Feature Aggregation Network for Video Object Detection

• DOI: 10.1109/tcsvt.2021.3094533
• 发表时间: 2022-12
• 期刊: IEEE Transactions on Circuits and Systems for Video Technology
• 影响因子: 8.4
• 作者: Liang Han;Pichao Wang;Zhaozheng Yin;Fan Wang;Hao Li