Object Comprehension via Robotic In-Hand Manipulation & Tactile Sensing

虽然人类在很大程度上依靠视觉来识别物体，但某些物体的属性只能通过使用触摸来可靠地确定，如粗糙度、弹性和重量分布。在操纵物品时，我们通常也会依靠视觉和触觉的结合：想象一下，你试图扔掉你在橱柜里找到的一根香蕉，但一触摸它，你就会立即意识到它的内部早已变成液体--然后你可能会试图小心地用香蕉的茎把它捡起来。有时，一个物体的形状甚至完全是通过触摸来确定的--想象一下，从装满其他随机物体的裤子口袋里找钥匙。我们不仅可以快速找到并抓住钥匙，而且还能够在将钥匙从口袋中取出之前迅速重新调整手中的钥匙。触觉感知帮助我们理解物体的机械属性，我们用这些属性来测试我们通过视觉和/或我们的记忆做出的假设，并最终在我们决定如何操纵物体方面发挥关键作用。这个博士项目专注于通过机器人手中操作(IHM)结合触觉感知来理解对象。我们所说的“物体理解”指的是类似于“3D物体重建”的过程(这是从视觉图像重建物体的形状和外观的过程)，但将触觉数据作为附加(或唯一)输入，并将机械特性的空间映射作为附加输出。最初的目标是开发一种定制的双手指、可变手掌宽度的机器人操作平台，能够在执行IHM的同时记录沿手指长度安装的传感器的触觉数据。机器人手的触觉数据和内部状态将被用来基于形状和机械特性构建对象的模型。对于博士学位的第一部分，研究工作将集中在开发适当的控制算法，以实现使用这种定制夹具进行高效的数据收集和重建。在PHD的第二部分中，将采用基于学习的方法来获取数据，并根据对人体系统中这类过程的认识来确定适当的触觉“探索技术”。将通过强化学习实施和训练稳健的自适应控制系统，使手/夹持器能够在通过IHM操纵对象的同时自适应地决定最优的探索动作。该系统的目标是最大限度地减少勘探时间，最大限度地提高勘探精度。很可能，为了促进开发更复杂的探查程序，我们将需要开发更复杂的带有额外手指和/或关节的夹持器。这第二部分的研究结果将探索IHM和触觉探索相结合用于物体理解和识别的可行性和潜力，并将为这一领域的未来研究建立一个基准。该项目的成果将包括利用触觉数据的新型自动化IHM对象探测和处理技术，这些技术可以帮助执行高级操作任务，如介绍性查找和对准口袋中的钥匙示例中的操作任务，并最终导致更安全、更合规、更高效和更多功能的机器人IHM。

Whilst humans largely rely on vision to recognise objects, there are certain object properties that can only be reliably determined through the use of touch, such as roughness, elasticity, and weight distribution. When manipulating objects, we often also rely on a combination of vision and touch: Imagine trying to throw away a banana you found in your cupboard, then upon touching it promptly realising the insides have long turned liquid - you would then perhaps attempt to carefully pick it up by the stem instead. Sometimes, the shape of an object is even determined solely through touch - Imagine searching for a key from your trouser pocket full of other random objects. Not only can we quickly locate and grab the key, we are also capable of swiftly re-aligning the key within our hand before pulling it out of the pocket. Haptic perception helps us comprehend the mechanical properties of objects, which we use to test against the assumptions we made through vision and/or our memory, and ultimately plays an essential role in how we decide to manipulate objects. This PhD project focuses on object comprehension through robotic In-Hand-Manipulation(IHM) in combination with tactile sensing. By 'object comprehension', we mean a process similar to "3D object reconstruction" (which is the process of reconstructing the shape and appearance of objects from visual images), but with haptic data as additional (or only) inputs and spatial mapping of mechanical properties as additional outputs. An initial goal is to develop a custom 2-finger, variable-width-palm, robotic manipulation platform that is able to perform IHM while recording tactile data from sensors mounted along the finger lengths. The tactile data, and internal states of the robotic hand will be used to construct a model of the object based on shape and mechanical properties. For this first part of the PhD, research efforts will focus on developing appropriate control algorithms to enable efficient data collection and reconstruction using this custom gripper. In the second part of the PhD, a learning-based approach to data acquisition techniques will be pursued, with the determination of appropriate haptic 'exploratory techniques' inspired by the recognition of such processes in human systems. A robust and adaptive control system will be implemented and trained through reinforcement learning, enabling the hand/gripper to adaptively decide on the optimal exploration actions whilst manipulating objects through IHM. This system has the goal of minimising exploration time and maximising exploration accuracy. It is likely that in order to facilitate the development of more complex exploratory procedures, we will need to develop a more complex gripper with additional fingers and/or articulation. The findings of this second part will explore the feasibility and potential of combining IHM and haptic exploration for objects comprehension and recognition, and will set a benchmark for future studies in this area. The results of this project will include novel automated IHM object exploration and handling techniques utilising tactile data, that can help perform advanced manipulation tasks such as that in the introductory finding-and-aligning-key-in-pocket example, and ultimately lead to safer, more compliant, more efficient, and more versatile robotic IHM.