Multimodal Sensory Integration and Control for Interactive Dexterous In-Hand Object Manipulation

用于交互式灵巧手持物体操纵的多模态感觉集成和控制

• 批准号: RGPIN-2015-05273
• 负责人: Jeon, Soo
• 金额: $ 2.48万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2017
• 资助国家: 加拿大
• 起止时间: 2017-01-01 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=638834
• 关键词: Multimodal; Sensory; Integration; Control; Interactive

Dexterous manipulation skills are one of the most remarkable features of human behavior. Even seemingly simple tasks of manipulating tools or objects embody feats of enormous complexity to process various sensory inputs and to make necessary adjustments all the while retrieving and updating motion vocabularies encoded in our brain. Traditionally, motion control for robot manipulators has pursued high-speed and high-accuracy trajectory tracking rather than natural, adaptable and dexterous movements. They enjoyed major success in factory environments (e.g. welding robots in automotive factories) but do not adequately apply to unstructured and dynamic environments that require interaction with random ’every-day’ objects or with humans. This proposal will attempt to tackle these issues by reaching beyond conventional methods to leverage sensor technologies, exploit intelligent decision making and control theories, and carefully observe how human hand and arm movements are coordinated during critical stages of object manipulation tasks. This research will take a bio-inspired engineering approach in that we will employ advanced sensing techniques to investigate how high-fidelity tactile data are integrated with proprioceptive (position, force/torque) and visual data to achieve ‘dexterity’ during object manipulation tasks, e.g. how finger-tip forces are regulated and how the hand and arm muscles are coordinated. The technical objectives of this research include 1) to develop integrated multimodal state estimator for simultaneous estimation of the robot, the object and the environment, 2) to investigate stereotyped patterns of tactile and proprioceptive sensory responses of humans during interactive object manipulation tasks, 3) to apply this knowledge to developing novel reflex (low-level, feedback) and high-level (motion planning, force/torque) controllers to enhance the sophistication of controlled motion of a robotic arm and hand during interaction periods, and 4) to develop learning control policies that enable the completion of the full cycle of manipulation phases at faster speed and with smoother transitions. Understanding human sensorimotor control and synthesizing artificial systems of similar capabilities will profoundly impact a wide range of applications including agile manufacturing, home automation, medical robots, prosthetics, rehabilitation engineering, and technologies that support the public welfare for aging population. Furthermore, the scientific findings and engineering achievements pursued through this research will place Canada at the forefront of research in key areas of intelligent control, smart sensory data processing, next generation of robotic manipulation and bio-mechatronics, which will also inspire and train the next generation of highly qualified talent who will contribute directly to innovation in these key sectors.

灵巧的操作技能是人类行为最显著的特征之一。即使是看似简单的操作工具或物体的任务，也包含着巨大的复杂性，要处理各种感官输入，并在检索和更新编码在我们大脑中的运动词汇的同时进行必要的调整。传统的机器人运动控制追求的是高速度、高精度的轨迹跟踪，而不是自然、灵活的运动。它们在工厂环境中取得了巨大的成功（例如汽车工厂中的焊接机器人），但不足以应用于需要与随机“日常”对象或人类交互的非结构化和动态环境。该提案将试图通过超越传统方法来解决这些问题，利用传感器技术，利用智能决策和控制理论，并仔细观察在物体操作任务的关键阶段，人手和手臂的运动是如何协调的。这项研究将采用生物启发的工程方法，我们将采用先进的传感技术来研究高保真触觉数据如何与本体感受（位置，力/扭矩）和视觉数据相结合，以实现对象操作任务期间的“灵活性”，例如指尖力如何调节以及手和手臂肌肉如何协调。本研究的技术目标包括：1）开发用于同时估计机器人、物体和环境的集成多模态状态估计器; 2）研究在交互式物体操作任务中人类的触觉和本体感受感觉反应的定型模式; 3）将这些知识应用于开发新的反射（低级别，反馈）和高级别（运动规划、力/扭矩）控制器，以增强在交互期间机器人臂和手的受控运动的复杂性，以及4）开发能够以更快的速度和更平滑的过渡完成操纵阶段的全周期的学习控制策略。理解人类感觉运动控制并合成具有类似能力的人工系统将深刻影响广泛的应用，包括敏捷制造，家庭自动化，医疗机器人，假肢，康复工程以及支持老龄化人口公共福利的技术。此外，通过这项研究所取得的科学发现和工程成就将使加拿大在智能控制、智能传感数据处理、下一代机器人操作和生物机电一体化等关键领域的研究中处于最前沿，这也将激励和培养下一代高素质人才，他们将直接为这些关键领域的创新做出贡献。