AI-driven Visual Servoing of Industrial Robotic Manufacturing Systems

近十年来，利用视觉信息进行运动控制的视觉伺服技术提高了工业机器人的灵巧性和精度。尽管对工业机器人的视觉伺服进行了大量的研究工作，但基于位置的视觉伺服（PBVS）和基于图像的视觉伺服（IBVS）策略都有各自的优缺点，需要加以解决以满足工业需求。例如，PBVS需要3D CAD对象模型，机器人运动学模型和相机校准模型的精确知识。但IBVS算法存在特征离开视场、收敛速度慢、局部最小等缺点。当前的PBVS和IBVS的成功很大程度上取决于先验信息或适当的图像特征或对象模型的可用性，有效的视觉伺服控制算法和学习机制。如今，先进的信息技术和人工智能（AI）的快速发展激发了机器人界开发各种智能机器人。人工智能技术具有巨大的潜力，可以使工业机器人在非结构化环境中工作，并为PBVS和IBVS处理上述问题。将人工智能技术融入到视觉伺服中，即人工智能驱动的视觉伺服，旨在使工业机器人具有灵活性和学习能力，提高制造性能，降低生产成本。人工智能驱动的视觉伺服的挑战在于融合所有可用的感官数据，如编码器、摄像头、力传感器、光学坐标测量机（CMM）和3D扫描仪，选择和重建图像特征，制定可以与未知环境交互并通过学习提高性能的控制策略等。在未来五年内，将探索人工神经网络、强化学习和机器学习等各种人工智能技术，以应对PBVS和IBVS中的上述挑战。该项目旨在开发具有智能、鲁棒性和适应性的工业机器人制造系统的ai驱动视觉伺服，实现真实环境下文件从设计到制造的无缝传递。最终目标是开发一个通过人工智能将学习技能嵌入工业机器人制造系统的系统框架。

Visual servoing, using the vision information for motion control, has improved the dexterity and accuracy of industrial robots in the past decade. Although a lot of research work has been carried out on the visual servoing of industrial robots, both the position-based visual servoing (PBVS) and image-based visual servoing (IBVS) strategies have its own advantages and disadvantages which need to be addressed to meet the industrial requirements. For example, PBVS requires a 3D CAD object model, a precise knowledge of the robot kinematics model and camera calibration model. While IBVS suffers the inherit drawbacks such as features leaving field of view (FOV), slow convergence and local minima, etc. The success of current PBVS and IBVS largely hinges on the prior information or availability of appropriate image features or the object model, effective visual servoing control algorithms and learning mechanism. Nowadays, the rapid development of advanced information technologies and artificial intelligence (AI) has inspired the robotics community to develop various intelligent robots. The AI technologies have the great potential to make the industrial robots work in the unstructured environment and deal with above mentioned issues for both PBVS and IBVS. Incorporating the AI technologies into the visual servoing, i.e., AI-driven visual servoing aims at equipping the industrial robots with the flexibility and learning abilities, improving the manufacturing performance and reducing the production costs. The challenges of AI-driven visual servoing lie in fusing all available sensory data such as encoder, camera, force sensor, optical coordinate measurement machines (CMM) and 3D scanner, selecting and reconstructing image features, and formulating control strategy that can interact with the unknown environment and can improve the performance through learning, etc. In the next five years, various AI technologies such as artificial neural network, reinforcement learning, and machine learning will be explored to meet the aforementioned challenges in both PBVS and IBVS. The project aims at developing AI-driven visual servoing of industrial robotic manufacturing systems with intelligence, robustness and adaptability, which can realize the seamless transfer of files from design to manufacturing in the real environment. The ultimate objective is to develop a systematic framework of embedding the learning skills via AI into industrial robotic manufacturing systems.