Object Comprehension via Robotic In-Hand Manipulation & Tactile Sensing

虽然人类在很大程度上依靠视觉来识别物体，但某些物体的属性只能通过触摸来可靠地确定，比如粗糙度、弹性和重量分布。在操纵物体时，我们也常常依赖于视觉和触觉的结合：想象一下，你试图扔掉你在橱柜里找到的一根香蕉，然后在触摸它时，你马上意识到里面早就变成液体了——然后你可能会试图小心地抓住它的茎，而不是把它捡起来。有时，一个物体的形状甚至仅仅是通过触摸来确定的——想象一下，从你满是其他随机物体的裤兜里寻找一把钥匙。我们不仅能快速找到并抓住钥匙，还能在把钥匙从口袋里拿出来之前迅速调整手中的钥匙。触觉感知帮助我们理解物体的机械特性，我们用它来测试我们通过视觉和/或记忆做出的假设，并最终在我们如何决定操纵物体方面发挥重要作用。这个博士项目的重点是通过机器人手操作（IHM）结合触觉感知来理解物体。通过“对象理解”，我们指的是类似于“3D对象重建”的过程（即从视觉图像中重建对象的形状和外观的过程），但将触觉数据作为附加（或仅）输入，并将机械属性的空间映射作为附加输出。最初的目标是开发一个定制的2指、可变手掌宽度的机器人操作平台，该平台能够执行IHM，同时记录安装在手指长度上的传感器的触觉数据。机器人手的触觉数据和内部状态将用于根据物体的形状和力学特性构建物体的模型。对于博士学位的第一部分，研究工作将集中在开发适当的控制算法上，以便使用这种定制夹具进行有效的数据收集和重建。在博士学位的第二部分，将追求基于学习的数据采集技术方法，并确定适当的触觉“探索技术”，这些技术的灵感来自于对人类系统中此类过程的识别。一个强大的自适应控制系统将通过强化学习实施和训练，使手/抓手能够自适应地决定最佳的探索行动，同时通过IHM操纵物体。该系统的目标是最小化勘探时间和最大化勘探精度。很可能为了促进更复杂的探查程序的发展，我们将需要开发一个更复杂的带有额外手指和/或关节的抓取器。第二部分的研究结果将探讨将IHM和触觉探索结合起来进行物体理解和识别的可行性和潜力，并将为该领域的未来研究奠定基础。该项目的结果将包括利用触觉数据的新型自动化IHM对象探索和处理技术，这可以帮助执行高级操作任务，例如在介绍性的寻找和对准口袋中的钥匙示例中，并最终导致更安全，更合规，更高效，更通用的机器人IHM。

Whilst humans largely rely on vision to recognise objects, there are certain object properties that can only be reliably determined through the use of touch, such as roughness, elasticity, and weight distribution. When manipulating objects, we often also rely on a combination of vision and touch: Imagine trying to throw away a banana you found in your cupboard, then upon touching it promptly realising the insides have long turned liquid - you would then perhaps attempt to carefully pick it up by the stem instead. Sometimes, the shape of an object is even determined solely through touch - Imagine searching for a key from your trouser pocket full of other random objects. Not only can we quickly locate and grab the key, we are also capable of swiftly re-aligning the key within our hand before pulling it out of the pocket. Haptic perception helps us comprehend the mechanical properties of objects, which we use to test against the assumptions we made through vision and/or our memory, and ultimately plays an essential role in how we decide to manipulate objects. This PhD project focuses on object comprehension through robotic In-Hand-Manipulation(IHM) in combination with tactile sensing. By 'object comprehension', we mean a process similar to "3D object reconstruction" (which is the process of reconstructing the shape and appearance of objects from visual images), but with haptic data as additional (or only) inputs and spatial mapping of mechanical properties as additional outputs. An initial goal is to develop a custom 2-finger, variable-width-palm, robotic manipulation platform that is able to perform IHM while recording tactile data from sensors mounted along the finger lengths. The tactile data, and internal states of the robotic hand will be used to construct a model of the object based on shape and mechanical properties. For this first part of the PhD, research efforts will focus on developing appropriate control algorithms to enable efficient data collection and reconstruction using this custom gripper. In the second part of the PhD, a learning-based approach to data acquisition techniques will be pursued, with the determination of appropriate haptic 'exploratory techniques' inspired by the recognition of such processes in human systems. A robust and adaptive control system will be implemented and trained through reinforcement learning, enabling the hand/gripper to adaptively decide on the optimal exploration actions whilst manipulating objects through IHM. This system has the goal of minimising exploration time and maximising exploration accuracy. It is likely that in order to facilitate the development of more complex exploratory procedures, we will need to develop a more complex gripper with additional fingers and/or articulation. The findings of this second part will explore the feasibility and potential of combining IHM and haptic exploration for objects comprehension and recognition, and will set a benchmark for future studies in this area. The results of this project will include novel automated IHM object exploration and handling techniques utilising tactile data, that can help perform advanced manipulation tasks such as that in the introductory finding-and-aligning-key-in-pocket example, and ultimately lead to safer, more compliant, more efficient, and more versatile robotic IHM.