AI-driven Visual Servoing of Industrial Robotic Manufacturing Systems

人工智能驱动的工业机器人制造系统视觉服务

• 批准号: RGPIN-2020-06813
• 负责人: Xie, WenFang
• 金额: $ 2.84万
• 依托单位: Concordia University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=741449
• 关键词: AI; driven; Visual; Servoing; Industrial

Visual servoing, using the vision information for motion control, has improved the dexterity and accuracy of industrial robots in the past decade. Although a lot of research work has been carried out on the visual servoing of industrial robots, both the position-based visual servoing (PBVS) and image-based visual servoing (IBVS) strategies have its own advantages and disadvantages which need to be addressed to meet the industrial requirements. For example, PBVS requires a 3D CAD object model, a precise knowledge of the robot kinematics model and camera calibration model. While IBVS suffers the inherit drawbacks such as features leaving field of view (FOV), slow convergence and local minima, etc. The success of current PBVS and IBVS largely hinges on the prior information or availability of appropriate image features or the object model, effective visual servoing control algorithms and learning mechanism. Nowadays, the rapid development of advanced information technologies and artificial intelligence (AI) has inspired the robotics community to develop various intelligent robots. The AI technologies have the great potential to make the industrial robots work in the unstructured environment and deal with above mentioned issues for both PBVS and IBVS. Incorporating the AI technologies into the visual servoing, i.e., AI-driven visual servoing aims at equipping the industrial robots with the flexibility and learning abilities, improving the manufacturing performance and reducing the production costs. The challenges of AI-driven visual servoing lie in fusing all available sensory data such as encoder, camera, force sensor, optical coordinate measurement machines (CMM) and 3D scanner, selecting and reconstructing image features, and formulating control strategy that can interact with the unknown environment and can improve the performance through learning, etc. In the next five years, various AI technologies such as artificial neural network, reinforcement learning, and machine learning will be explored to meet the aforementioned challenges in both PBVS and IBVS. The project aims at developing AI-driven visual servoing of industrial robotic manufacturing systems with intelligence, robustness and adaptability, which can realize the seamless transfer of files from design to manufacturing in the real environment. The ultimate objective is to develop a systematic framework of embedding the learning skills via AI into industrial robotic manufacturing systems.

视觉伺服利用视觉信息进行运动控制，在过去的十年中提高了工业机器人的灵活性和准确性。虽然人们对工业机器人的视觉伺服进行了大量的研究工作，但基于位置的视觉伺服（PBVS）和基于图像的视觉伺服（IBVS）策略各有优缺点，需要加以解决以满足工业需求。例如，PBVS需要3D CAD对象模型、机器人运动学模型和相机校准模型的精确知识。而IBVS具有固有的缺点，如功能离开视场（FOV），收敛速度慢，局部极小值等，目前的PBVS和IBVS的成功在很大程度上取决于先验信息或适当的图像特征或对象模型，有效的视觉伺服控制算法和学习机制的可用性。 当今，先进的信息技术和人工智能（AI）的快速发展激发了机器人界开发各种智能机器人的灵感。人工智能技术具有很大的潜力，使工业机器人在非结构化环境中工作，并处理上述问题的PBVS和IBVS。将人工智能技术应用于视觉伺服，即，人工智能驱动的视觉伺服旨在使工业机器人具有灵活性和学习能力，提高制造性能，降低生产成本。人工智能驱动的视觉伺服的挑战在于融合编码器、摄像头、力传感器、光学坐标测量机（CMM）和3D扫描仪等所有可用的传感数据，选择和重建图像特征，制定能够与未知环境交互并通过学习提高性能的控制策略等，未来五年，人工神经网络、强化学习和机器学习将被探索，以应对PBVS和IBVS中的上述挑战。该项目旨在开发具有智能性、鲁棒性和适应性的工业机器人制造系统的AI驱动视觉伺服，可以在真实的环境中实现从设计到制造的文件无缝传输。最终目标是开发一个系统框架，通过人工智能将学习技能嵌入工业机器人制造系统。