Keypoints-Based Adaptive Visual Servoing for Control of Robotic Manipulators in Configuration Space
Keypoints-Based Adaptive Visual Servoing for Control of Robotic Manipulators in Configuration Space
复制标题
DOI：
10.1109/iros55552.2023.10342503
复制
发表时间：
2023-10
期刊：
影响因子：
--
通讯作者：
Sreejani Chatterjee;Abhay C. Karade;Abhinav Gandhi;B. Çalli
中科院分区：
文献类型：
--
作者：
Sreejani Chatterjee;Abhay C. Karade;Abhinav Gandhi;B. Çalli
本文提出了一种纯粹利用机器人的自然特征在位形空间中控制机器人的视觉伺服方法。我们首先创建了一个数据收集管道，该管道使用摄像机内部、外部和正向运动学来生成图像空间中机器人关节位置（关键点）的2D投影。使用这个管道，我们能够收集大量的真实机器人数据，我们使用这些数据来训练实时关键点检测器。从训练模型中推断出的关键点被用作自适应视觉伺服方案中的控制特征，该方案在运行时估计与关键点和关节速度的变化相关的雅可比矩阵。我们比较了这种方法的2D配置控制性能的基于视觉伺服的方法（唯一的其他算法，纯粹基于视觉的配置空间视觉伺服），并证明了关键点提供更强大的和更少的噪声功能，从而导致更好的瞬态响应。我们还展示了第一个基于视觉的3D配置空间控制的结果在文献中，并讨论其局限性。我们的数据收集管道可在https://github.com/JaniC-WPI/KPDataGenerator.git上获得，可用于收集图像数据集并为各种机器人和环境训练实时关键点检测器。
This paper presents a visual servoing method for controlling a robot in the configuration space by purely using its natural features. We first created a data collection pipeline that uses camera intrinsics, extrinsics, and forward kinematics to generate 2D projections of a robot's joint locations (keypoints) in image space. Using this pipeline, we are able to collect large sets of real-robot data, which we use to train realtime keypoint detectors. The inferred keypoints from the trained model are used as control features in an adaptive visual servoing scheme that estimates, in runtime, the Jacobian relating the changes of the keypoints and joint velocities. We compared the 2D configuration control performance of this method to the skeleton-based visual servoing method (the only other algorithm for purely vision-based configuration space visual servoing), and demonstrated that the keypoints provide more robust and less noisy features, which result in better transient response. We also demonstrate the first vision-based 3D configuration space control results in the literature, and discuss its limitations. Our data collection pipeline is available at https://github.com/JaniC-WPI/KPDataGenerator.git which can be utilized to collect image datasets and train realtime keypoint detectors for various robots and environments.