ERI: High-performance Human-robot Collaborative Manufacturing Enabled by Integrated Multimodal Teaching, Learning, Prediction and Interaction

ERI：通过集成多模态教学、学习、预测和交互实现高性能人机协作制造

• 批准号: 2138351
• 负责人: Weitian Wang
• 金额: $ 20万
• 依托单位: Montclair State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-02-01 至 2025-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2138351&HistoricalAwards=false
• 关键词: ERI; performance; Human; robot; Collaborative

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Robotics is one of critical technologies in advancing the manufacturing industry, because of its potential to heighten the efficiency in the productivity and part quality. Traditional industrial robots are fenced off from human workers on production lines; on the contrary, collaborative robots are not, making them capable of democratizing manufacturing industries with dynamic customer demands and high flexibility. Currently, however, most collaborative robots are programmed by conventional off-line coding for specific manufacturing applications, e.g., assembly. Then the robots collaborate with human partners via predefined workflows to perform simply repetitive tasks. Such a static human-robot collaborative manufacturing process may be degraded, if any work environment settings or tasks are ever changed. To address this challenge and advance human-robot collaborative manufacturing, this Engineering Research Initiation (ERI) award will develop a competitive solution to train robots not only to be effectively programed by learning from human demonstrations, but also to actively assist human partners to jointly accomplish manufacturing tasks. The research will contribute toward fundamental engineering research on human-robot collaboration in advanced manufacturing. This project will offer students at Montclair State University, which has a diverse student body from underrepresented groups, with the latest robotics training and research, which will diversify the future workforce and potentially benefit the US industry. In addition, this project will launch robotics workshops with cutting-edge hands-on activities for local K-12 schools, especially from underserved districts.The goal of this project is to develop a teaching-learning-prediction-collaboration framework for robots to proactively learn from human demonstrations, predict human intentions, and collaborate with humans in collaborative manufacturing tasks. The major questions to be solved include the following: (i) how can a human-robot collaborative manufacturing process be mathematically described and can robots learn task knowledge from human demonstrations, (ii) how can robots assist human partners based on the prediction of human intentions in the collaboration process, and (iii) how can the framework be validated in human-robot collaborative manufacturing tasks? To fill the knowledge gaps, the human-robot collaboration will be parameterized through a Markov Decision Process and develop a multimodal-information-based approach for robots to learn task customization and human working preference from human partners’ demonstrations in collaborative manufacturing environments. Further, computational human intention prediction and human-robot collaboration models will be developed for robots to leverage the learned strategies to proactively predict human partners’ upcoming intentions and assist humans in shared tasks. Moreover, user studies will be conducted to evaluate the effectiveness of the approaches in collaboration quality improvement by applying findings to real-world human-robot collaborative tasks in advanced manufacturing contexts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项全部或部分由2021年美国救援计划法案（公法117-2）资助。机器人技术是推动制造业发展的关键技术之一，因为它具有提高生产效率和零件质量的潜力。传统的工业机器人与生产线上的人类工人隔离开来;相反，协作机器人则不然，这使得它们能够以动态的客户需求和高度灵活性实现制造业的民主化。然而，目前，大多数协作机器人都是通过传统的离线编码来编程的，用于特定的制造应用，例如，组装件.然后，机器人通过预定义的工作流程与人类合作，执行简单的重复性任务。如果任何工作环境设置或任务发生变化，则这种静态的人机协作制造过程可能会降级。为了应对这一挑战并推进人机协作制造，该工程研究启动（ERI）奖项将开发一种有竞争力的解决方案，以训练机器人不仅通过从人类演示中学习来有效编程，而且还积极协助人类合作伙伴共同完成制造任务。该研究将有助于先进制造中人机协作的基础工程研究。该项目将为蒙特克莱尔州立大学的学生提供最新的机器人培训和研究，这将使未来的劳动力多样化，并可能使美国工业受益。此外，该项目还将为当地的K-12学校，特别是服务水平低下地区的学校，举办机器人研讨会，并开展尖端的实践活动。该项目的目标是开发一个教-学-预测-协作框架，使机器人能够主动从人类演示中学习，预测人类意图，并在协作制造任务中与人类协作。要解决的主要问题包括以下几个方面：（一）如何可以在数学上描述的人与机器人的协同制造过程中，机器人可以学习任务知识，从人类的演示，（二）机器人如何帮助人类的合作伙伴的基础上预测的人的意图在协作过程中，以及（iii）如何可以在人与机器人的协同制造任务的框架进行验证？为了填补知识空白，人-机器人协作将通过马尔可夫决策过程参数化，并开发一个多模态信息为基础的方法，机器人学习任务定制和人类的工作偏好，从人类合作伙伴的示范，在协同制造环境。此外，将为机器人开发计算人类意图预测和人类-机器人协作模型，以利用学习到的策略来主动预测人类合作伙伴即将到来的意图，并协助人类完成共享任务。此外，还将进行用户研究，通过将研究结果应用于先进制造环境中的真实人机协作任务，评估协作质量改进方法的有效性。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Ready or Not? A Robot-Assisted Crop Harvest Solution in Smart Agriculture Contexts

• DOI: 10.1109/smartcomp58114.2023.00088
• 发表时间: 2023-06
• 期刊: 2023 IEEE International Conference on Smart Computing (SMARTCOMP)
• 作者: Thai Thao Nguyen;Jesse Parron;Omar Obidat;A. R. Tuininga;Weitian Wang

JUST TELL ME: A Robot-assisted E-health Solution for People with Lower-extremity Disability

请告诉我：针对下肢残障人士的机器人辅助电子健康解决方案

• DOI: 10.1109/icara56516.2023.10125947
• 发表时间: 2023
• 期刊: Robotics and Applications (ICARA
• 作者: Coutras, Alexander;Obidat, Omar;Zhu, Michelle;Wang, Weitian
• 通讯作者: Wang, Weitian

Understanding Dynamic Human Intentions to Enhance Collaboration Performance for Human-Robot Partnerships

了解人类的动态意图以增强人机伙伴关系的协作性能

• 发表时间: 2023
• 期刊: IEEE
• 作者: Jacoby I., Parron J.;Wang W.
• 通讯作者: Wang W.

Development and Implementation of an AI-Embedded and ROS-Compatible Smart Glove System in Human-Robot Interaction

• DOI: 10.1109/mass56207.2022.00103
• 发表时间: 2022-10
• 期刊: 2022 IEEE 19th International Conference on Mobile Ad Hoc and Smart Systems (MASS)
• 作者: Laury Rodriguez;Zofia Przedworska;Omar Obidat;Jesse Parron;Weitian Wang

Development of a Teaching-Learning-Prediction-Collaboration Model for Human-Robot Collaborative Tasks

人机协作任务的教-学-预测-协作模型的开发

• 发表时间: 2023
• 期刊: and Intelligent Systems
• 作者: Obidat, O.;Parron, J.;Li, R.;Rodano, J.;Wang, W.
• 通讯作者: Wang, W.