Master of Puppets: Multi-modal Robot Activity Segmentation from Teleoperated Demonstrations
Master of Puppets: Multi-modal Robot Activity Segmentation from Teleoperated Demonstrations
复制标题
DOI：
10.1109/icdl53763.2022.9962193
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Claudio Coppola;L. Jamone
中科院分区：
文献类型：
--
作者：
Claudio Coppola;L. Jamone
在非结构化环境(例如，轻型制造、极端环境)中为复杂任务编程机器人不能仅通过分析方法完成。从远程操作的人类演示中学习是一种很有前途的方法，可以减少编程负担，获得更有效的控制器。但是，需要将录制的演示分解为原子操作，以便于表示所需的行为，这在现实世界中可能非常具有挑战性。在这项研究中，我们提出了一种方法，利用从机器人运动和触觉数据中提取的特征来自动从遥操作序列中分割出原子动作。我们创建了一个公开可用的数据集，其中包含在单一对象和杂乱情况下对三个不同对象进行机器人拾取和放置的演示。我们使用定制的遥操作系统，将用户的手和指尖姿势映射到配备触觉传感器的三指灵巧机器人手中。我们的发现表明，所提出的特征集概括了同一对象不同情节中的活动以及大小相似的项目之间的活动。此外，他们还指出，触觉感知有助于更好地识别演示中的活动。
Programming robots for complex tasks in unstructured settings (e.g., light manufacturing, extreme environments) cannot be accomplished solely by analytical methods. Learning from teleoperated human demonstrations is a promising approach to decrease the programming burden and to obtain more effective controllers. However, the recorded demonstrations need to be decomposed into atomic actions to facilitate the representation of the desired behaviour, which can be very challenging in real-world settings. In this study, we propose a method that uses features extracted from robot motion and tactile data to automatically segment atomic actions from a teleoperation sequence. We created a publicly available dataset with demonstrations of robotic pick-and-place of three different objects in single-object and cluttered situations. We use a custom-built teleoperation system that maps the user’s hand and fingertips poses into a three-fingered dexterous robot hand equipped with tactile sensors. Our findings suggest that the proposed feature set generalises the activities in different episodes of the same object and between items of similar size. Furthermore, they suggest that tactile sensing contributes to higher performance in recognising activities within demonstrations.