ERI: High-performance Human-robot Collaborative Manufacturing Enabled by Integrated Multimodal Teaching, Learning, Prediction and Interaction

该奖项的全部或部分资金来源于《2021 年美国救援计划法案》（公法 117-2）。机器人是推动制造业发展的关键技术之一，因为它具有提高生产力效率和零件质量的潜力。传统的工业机器人在生产线上与人类工人隔离；相反，协作机器人则不然，这使得它们能够通过动态的客户需求和高度灵活性实现制造业的民主化。然而，目前大多数协作机器人都是通过针对特定制造应用（例如装配）的传统离线编码进行编程的。然后，机器人通过预定义的工作流程与人类合作伙伴协作，执行简单的重复性任务。如果任何工作环境设置或任务发生改变，这种静态的人机协作制造过程可能会降级。为了应对这一挑战并推进人机协作制造，该工程研究启动（ERI）奖将开发一种有竞争力的解决方案来训练机器人，不仅通过学习人类演示来进行有效编程，而且还积极协助人类合作伙伴共同完成制造任务。该研究将为先进制造中人机协作的基础工程研究做出贡献。该项目将为蒙特克莱尔州立大学的学生提供最新的机器人培训和研究，该大学拥有来自代表性不足群体的多元化学生群体，这将使未来的劳动力多样化，并可能使美国工业受益。此外，该项目还将为当地的 K-12 学校，特别是服务水平较低的地区的学校举办机器人研讨会，提供前沿的实践活动。该项目的目标是开发一个教-学-预测-协作框架，让机器人能够主动学习人类演示，预测人类意图，并在协作制造任务中与人类协作。需要解决的主要问题包括：（i）如何用数学方法描述人机协同制造过程以及机器人能否从人类演示中学习任务知识；（ii）机器人如何根据协作过程中人类意图的预测来协助人类伙伴；（iii）如何在人机协同制造任务中验证该框架？为了填补知识空白，人机协作将通过马尔可夫决策过程进行参数化，并开发一种基于多模态信息的方法，让机器人从协作制造环境中的人类合作伙伴的演示中学习任务定制和人类工作偏好。此外，还将为机器人开发计算人类意图预测和人机协作模型，以利用学习到的策略主动预测人类伙伴即将到来的意图并协助人类完成共同任务。此外，还将进行用户研究，通过将研究结果应用于先进制造环境中的现实世界人机协作任务，来评估协作质量改进方法的有效性。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Robotics is one of critical technologies in advancing the manufacturing industry, because of its potential to heighten the efficiency in the productivity and part quality. Traditional industrial robots are fenced off from human workers on production lines; on the contrary, collaborative robots are not, making them capable of democratizing manufacturing industries with dynamic customer demands and high flexibility. Currently, however, most collaborative robots are programmed by conventional off-line coding for specific manufacturing applications, e.g., assembly. Then the robots collaborate with human partners via predefined workflows to perform simply repetitive tasks. Such a static human-robot collaborative manufacturing process may be degraded, if any work environment settings or tasks are ever changed. To address this challenge and advance human-robot collaborative manufacturing, this Engineering Research Initiation (ERI) award will develop a competitive solution to train robots not only to be effectively programed by learning from human demonstrations, but also to actively assist human partners to jointly accomplish manufacturing tasks. The research will contribute toward fundamental engineering research on human-robot collaboration in advanced manufacturing. This project will offer students at Montclair State University, which has a diverse student body from underrepresented groups, with the latest robotics training and research, which will diversify the future workforce and potentially benefit the US industry. In addition, this project will launch robotics workshops with cutting-edge hands-on activities for local K-12 schools, especially from underserved districts.The goal of this project is to develop a teaching-learning-prediction-collaboration framework for robots to proactively learn from human demonstrations, predict human intentions, and collaborate with humans in collaborative manufacturing tasks. The major questions to be solved include the following: (i) how can a human-robot collaborative manufacturing process be mathematically described and can robots learn task knowledge from human demonstrations, (ii) how can robots assist human partners based on the prediction of human intentions in the collaboration process, and (iii) how can the framework be validated in human-robot collaborative manufacturing tasks? To fill the knowledge gaps, the human-robot collaboration will be parameterized through a Markov Decision Process and develop a multimodal-information-based approach for robots to learn task customization and human working preference from human partners’ demonstrations in collaborative manufacturing environments. Further, computational human intention prediction and human-robot collaboration models will be developed for robots to leverage the learned strategies to proactively predict human partners’ upcoming intentions and assist humans in shared tasks. Moreover, user studies will be conducted to evaluate the effectiveness of the approaches in collaboration quality improvement by applying findings to real-world human-robot collaborative tasks in advanced manufacturing contexts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.