A Biomimetic Approach based on Tactile Sensing for Stable Grasp and Manipulation using Biomechatronic Hand Prostheses and Assistive Robots

触觉感知是人类如何与环境互动的基本组成部分。随着上半身失去一条肢体，截肢者也失去了使用触觉探索、理解和与周围世界互动的能力。各种截肢者使用的主动手假体只能提供对对象处理的粗略控制；用户在不仔细查看对象或工具的情况下无法调整其手的运动来抓住对象或处理工具。为了提高截肢者对假肢的接受度，使用者应该能够进行增强的、直观的运动控制，并同时用假肢进行感觉。当在未知环境中工作时，触觉敏感度与其他感官来源一起，对于安全有效的物理交互和执行复杂任务变得至关重要。从长远来看，我的目标是开发一种具有成本效益的生物机电假手，通过结合触觉感知和触觉反馈，能够稳定地抓取物体和手中操作。这种假体将在日常生活的各种活动中促进与环境的安全互动，从而理想地帮助用户。受人类抓取、处理和移动手中物体的方案的启发，这项发现赠款将在5年的短期窗口中使用，以创建一个使用触觉传感器的机械臂抓取和操纵物体的框架。我假设，理解和模仿人类将触觉信息融入物体处理过程的方式，将导致设计出一种更有效的计划和控制方法。首先，我们将开发一个机器学习模型，从演示中学习非截肢者的触觉敏感度如何与视觉协同工作，以及它如何与肌肉协同作用相对应，从而为抓取和操纵物体创造运动动作。然后，我们将制定一个指数来评估使用触觉信息的抓取的稳定性，而不需要事先了解对象和环境。利用这个指标，我们随后将设计一个强化学习模型，通过从成功和失败的尝试中学习来微调控制模型。最后，将使用开发的人工智能模型来制作触觉界面的原型，以控制辅助机械手。最后，我们将通过对训练集和一组未知的真实世界对象执行抓取和操纵任务来验证所提出的框架的有效性。触觉接口和控制算法将直接使患有神经肌肉疾病和上肢截肢的个人在日常活动中受益，这将有望提升加拿大在辅助和假肢机器人领域的国际领先地位。

Tactile sensing is a fundamental component of how humans interact with their environment. With the loss of a limb in the upper body, the amputees also lose their ability to explore, understand and interact with the surrounding world using their sense of touch. The active hand prostheses used by a variety of amputees are only able to provide crude control over object handling; The users cannot adjust their hand movement for grasping an object or handling a tool without closely looking at it. To increase the acceptance of artificial limbs by amputees, the users should be able to perform enhanced, intuitive motor control and simultaneously feel with their prosthetic limbs. When operating in an unknown environment, tactile sensitivity, in conjunction with other sensory sources, becomes vital for a safe and effective physical interaction and performing complex tasks. In the long-term, I aim to develop a cost-efficient biomechatronic prosthetic hand capable of stable object grasping and in-hand manipulation by incorporating tactile sensing and haptic feedback. This prosthesis will ideally help users through facilitating safe interactions with the environment in various activities of daily living. Inspired by the schemes that humans employ for grasping, handling and moving objects in their hand, this Discovery Grant will be used in a short-term window of 5 years to create a framework for object grasping and manipulation by robotic arms using tactile sensors. I hypothesize that understanding and mimicking the human way of incorporating information from the sense of touch into the process of object handling, will result in devising a more efficient planning and control approach. First, we will develop a machine learning model to learn, from demonstrations, how tactile sensitivity in non-amputee subjects works in collaboration with vision, and how it corresponds to muscle synergies creating motor actions for grasping and manipulating objects. Then, we will formulate an index to assess the stability of a grasp using tactile information without having prior knowledge about the objects and the environment. Using this index, we will subsequently design a reinforcement learning model to fine-tune the control model by learning from successful and failed attempts. Lastly, a haptic interface will be prototyped to control an assistive robotic hand using the developed AI models. At the end, we will validate the effectiveness of the proposed framework by performing grasping and manipulation tasks with both the training set and a set of unknown real-world objects. The haptic interface and control algorithm will directly benefit individuals with neuromuscular disorders and upper-limb amputation in their day-to-day activities, which will hopefully elevate Canada's place as an international leader in the field of assistive and prosthetic robotics.