Physics-Informed Reinforcement Learning for Tactile Manipulation

用于触觉操作的物理强化学习

• 批准号: 2614951
• 金额: --
• 依托单位: University of Bristol
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2021
• 资助国家: 英国
• 起止时间: 2021 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2614951
• 关键词: Physics; Informed; Reinforcement; Learning; Tactile

Object manipulation in unstructured and unfamiliar environments is a powerful tool that can greatly increase the value of robotics. However, research effort in this area has been largely focused on using vision as input for control whilst little attention has been given to the application of touch. Tactile sensing is key for achieving human-level dexterity. Measurements such as contact compliance, contact location, shear, pressure, slip, and other important mechanical properties of contact, that would otherwise be inaccessible through vision, can be obtained even when objects are partially occluded. In my research, I hope to harness this power of touch to develop control frameworks to support robots' interaction with the real-world. To fully realise the value of autonomous manipulation, robots need to operate under a wide range of pervasive and unsystematic scenarios that it nor its designers have foreseen before, many of which can be impossible to model. This calls for advanced control frameworks with high versatility and generality that is also robust enough to deal with a range of unpredictable situations. Reinforcement learning (RL) is a popular learning algorithm that can allow robots to learn a complex controller through trial and error. Adopting such technique would simplify the controller design problem, avoiding the need to develop specialised controller for every scenario in our unstructured world and significantly reducing the research effort required to achieve human-level dexterity. However, reinforcement learning can be extremely fragile. The learning process can be very sensitive to certain design parameters and small deviations from expected behaviour can lead to catastrophic failures. It also suffers from high sample complexity which makes real-world learning unfeasible. These drawbacks have meant that much of the recent advances in reinforcement learning for robotics have involved experimenting in simulations before transferring to the real-world, creating many sim-to-real problems. Model-based reinforcement learning is a more recent approach that aims to tackle these shortcomings by first learning a representation of the transition dynamics before using it to derive the optimal controller. This has been shown to be more effective for robot control where high-capacity models such as neural networks or probabilistic models can be used to solve complex control problems with reduced number of interactions. This not only makes real-world learning possible, but the existence of a model can improve robustness amongst many other benefits. The application of model-based reinforcement learning to tactile robotics has very much been underexplored and I believe this combination could massively progress the capabilities of tactile manipulation. My research will aim to further this vision, with a focus on safe and efficient real-world learning and deployment. I hope to develop methodologies for synthesising controller that is; sample efficient in the online learning process to exhibit adaptive behaviour, robust in deployment to be able to deal with unpredictable situations, and also remain general in formulation such that it can be applied to a range of tactile manipulation tasks. A potential direction I hope to explore is to incorporate physics-priors to guide the learning process which can potentially mitigate the major efficiency issues associated with the trial-and-error nature of reinforcement learning. This research falls within the EPSRC artificial intelligence and robotics area.

在非结构化和陌生环境中进行对象操作是一个强大的工具，可以大大增加机器人技术的价值。然而，该领域的研究工作主要集中在使用视觉作为控制输入，而很少关注触摸的应用。触觉传感是实现人类敏捷程度的关键。即使物体被部分遮挡，也可以获得接触顺应性、接触位置、剪切力、压力、滑移和其他重要的接触机械性能等测量结果，而这些测量结果是通过视觉无法获得的。在我的研究中，我希望利用触摸的力量来开发控制框架来支持机器人与现实世界的交互。为了充分实现自主操纵的价值，机器人需要在其及其设计者之前预见到的各种普遍且非系统的场景下进行操作，其中许多场景是无法建模的。这就需要具有高度通用性和通用性的先进控制框架，并且足够强大，可以处理一系列不可预测的情况。强化学习（RL）是一种流行的学习算法，可以让机器人通过反复试验来学习复杂的控制器。采用这种技术将简化控制器设计问题，避免为非结构化世界中的每种场景开发专门的控制器，并显着减少实现人类水平的灵活性所需的研究工作。然而，强化学习可能极其脆弱。学习过程可能对某些设计参数非常敏感，与预期行为的微小偏差可能会导致灾难性的失败。它还面临样本复杂性高的问题，这使得现实世界的学习变得不可行。这些缺点意味着机器人强化学习的最新进展大多涉及在转移到现实世界之前进行模拟实验，从而产生了许多模拟到现实的问题。基于模型的强化学习是一种较新的方法，旨在通过首先学习过渡动态的表示，然后使用它来导出最佳控制器来解决这些缺点。事实证明，这对于机器人控制更为有效，其中可以使用神经网络或概率模型等高容量模型来解决复杂的控制问题，同时减少交互次数。这不仅使现实世界的学习成为可能，而且模型的存在可以提高鲁棒性以及许多其他好处。基于模型的强化学习在触觉机器人中的应用尚未得到充分探索，我相信这种组合可以极大地提高触觉操纵的能力。我的研究旨在进一步推进这一愿景，重点关注安全高效的现实世界学习和部署。我希望开发综合控制器的方法，即：在线学习过程中的样本有效，以表现出适应性行为，部署稳健，能够处理不可预测的情况，并且在公式中保持通用性，使其可以应用于一系列触觉操作任务。我希望探索的一个潜在方向是结合物理先验来指导学习过程，这可能会减轻与强化学习的试错性质相关的主要效率问题。这项研究属于 EPSRC 人工智能和机器人领域。