How to Select and Use Tools? : Active Perception of Target Objects Using Multimodal Deep Learning
How to Select and Use Tools? : Active Perception of Target Objects Using Multimodal Deep Learning
复制标题
如何选择和使用工具？
DOI：
10.1109/lra.2021.3062004
复制
发表时间：
2021
影响因子：
5.2
通讯作者：
Shigeki Sugano
中科院分区：
文献类型：
--
作者：
Namiko Saito;Tetsuya Ogata;Satoshi Funabashi;Hiroki Mori;Shigeki Sugano
选择适当的工具并在执行日常任务时使用它们是将机器人引入家庭应用的关键功能。然而，在以前的研究中，对目标对象的适应性是有限的，因此很难相应地改变工具和调整动作。为了用工具操纵各种物体，机器人必须理解工具的功能，并识别物体的特征，以识别工具-物体-动作的关系。我们专注于使用多模态感觉运动数据的主动感知，而机器人与物体进行交互，并允许机器人识别它们的外在和内在特征。我们构建了一个深度神经网络（DNN）模型，该模型学习识别对象特征，获取工具-对象-动作关系，并生成用于工具选择和处理的运动。作为工具使用情况的示例，机器人执行配料转移任务，使用翻勺或勺子将配料从锅转移到碗。结果证实，即使目标成分是未知的，机器人识别对象的特性和服务。我们还研究了图像，力和触觉数据的贡献，并表明学习各种多模态信息的结果丰富的感知工具的使用。
Selection of appropriate tools and use of them when performing daily tasks is a critical function for introducing robots for domestic applications. In previous studies, however, adaptability to target objects was limited, making it difficult to accordingly change tools and adjust actions. To manipulate various objects with tools, robots must both understand tool functions and recognize object characteristics to discern a tool–object–action relation. We focus on active perception using multimodal sensorimotor data while a robot interacts with objects, and allow the robot to recognize their extrinsic and intrinsic characteristics. We construct a deep neural networks (DNN) model that learns to recognize object characteristics, acquires tool–object–action relations, and generates motions for tool selection and handling. As an example tool-use situation, the robot performs an ingredients transfer task, using a turner or ladle to transfer an ingredient from a pot to a bowl. The results confirm that the robot recognizes object characteristics and servings even when the target ingredients are unknown. We also examine the contributions of images, force, and tactile data and show that learning a variety of multimodal information results in rich perception for tool use.