Multimodal Sensory Integration and Control for Interactive Dexterous In-Hand Object Manipulation

灵巧的操作技巧是人类行为最显著的特征之一。即使是看似简单的操作工具或物体的任务，在检索和更新我们大脑中编码的运动词汇的同时，也要处理各种感觉输入并进行必要的调整，这一壮举实际上是极其复杂的。传统的机器人运动控制追求的是高速、高精度的轨迹跟踪，而不是自然、自适应、灵巧的运动。它们在工厂环境中获得了巨大成功(例如汽车工厂中的焊接机器人)，但不能充分应用于需要与随机的“日常”物体或与人交互的非结构化和动态环境。这项提案将试图通过超越传统方法来利用传感器技术，利用智能决策和控制理论，并仔细观察在物体操纵任务的关键阶段，人类的手和手臂的运动是如何协调的，以解决这些问题。这项研究将采用生物启发的工程方法，我们将使用先进的传感技术来研究高保真触觉数据如何与本体感知(位置、力/扭矩)和视觉数据相结合，以在对象操作任务中实现“灵活性”，例如如何调节指尖的力以及如何协调手和手臂的肌肉。这项研究的技术目标包括1)开发集成的多模式状态估计器，用于同时估计机器人、物体和环境；2)研究交互物体操作任务中人类触觉和本体感觉反应的刻板模式；3)将这一知识应用于开发新的反射(低级反馈)和高级(运动规划、力/力矩)控制器，以提高交互期间机械臂和手的受控运动的复杂性；以及4)开发学习控制策略，使操作阶段的整个周期能够以更快的速度和更平稳的过渡完成。了解人类感觉运动控制并合成类似能力的人工系统将深刻影响广泛的应用，包括敏捷制造、家庭自动化、医疗机器人、假肢、康复工程以及支持人口老龄化公共福利的技术。此外，通过这项研究取得的科学发现和工程成就将使加拿大处于智能控制、智能传感数据处理、下一代机器人操纵和生物机电一体化等关键领域研究的前沿，这也将激励和培训下一代高素质人才，他们将直接为这些关键领域的创新做出贡献。**

Dexterous manipulation skills are one of the most remarkable features of human behavior. Even seemingly simple tasks of manipulating tools or objects embody feats of enormous complexity to process various sensory inputs and to make necessary adjustments all the while retrieving and updating motion vocabularies encoded in our brain. Traditionally, motion control for robot manipulators has pursued high-speed and high-accuracy trajectory tracking rather than natural, adaptable and dexterous movements. They enjoyed major success in factory environments (e.g. welding robots in automotive factories) but do not adequately apply to unstructured and dynamic environments that require interaction with random 'every-day' objects or with humans. This proposal will attempt to tackle these issues by reaching beyond conventional methods to leverage sensor technologies, exploit intelligent decision making and control theories, and carefully observe how human hand and arm movements are coordinated during critical stages of object manipulation tasks. This research will take a bio-inspired engineering approach in that we will employ advanced sensing techniques to investigate how high-fidelity tactile data are integrated with proprioceptive (position, force/torque) and visual data to achieve `dexterity' during object manipulation tasks, e.g. how finger-tip forces are regulated and how the hand and arm muscles are coordinated. The technical objectives of this research include 1) to develop integrated multimodal state estimator for simultaneous estimation of the robot, the object and the environment, 2) to investigate stereotyped patterns of tactile and proprioceptive sensory responses of humans during interactive object manipulation tasks, 3) to apply this knowledge to developing novel reflex (low-level, feedback) and high-level (motion planning, force/torque) controllers to enhance the sophistication of controlled motion of a robotic arm and hand during interaction periods, and 4) to develop learning control policies that enable the completion of the full cycle of manipulation phases at faster speed and with smoother transitions. Understanding human sensorimotor control and synthesizing artificial systems of similar capabilities will profoundly impact a wide range of applications including agile manufacturing, home automation, medical robots, prosthetics, rehabilitation engineering, and technologies that support the public welfare for aging population. Furthermore, the scientific findings and engineering achievements pursued through this research will place Canada at the forefront of research in key areas of intelligent control, smart sensory data processing, next generation of robotic manipulation and bio-mechatronics, which will also inspire and train the next generation of highly qualified talent who will contribute directly to innovation in these key sectors.**