Integrated Machine Learning and Control for Synthesis of Dexterous Manipulation Skills

协作机器人的兴起预示着一个新时代的到来，在我们的日常生活中，无论是家庭还是工作场所，机器人都可以与人类共存。机器学习的最新进展迅速改变了我们设计自主系统的方式，即使在以前被认为过于复杂而无法取代人类的情况下（例如驾驶汽车或下围棋）。虽然这些数据驱动的工具为基于知识的推理（如分析图像或音频数据）提供了希望，但如何将这些技术转移到机械设备（如机械手臂和手）的复杂电机控制中仍然是一个问题。人类的四肢远远优于人造的四肢；这并不在于反复执行预编程任务的能力和准确性，而是在于复杂和不可预测任务中熟练动作的丰富性、适应性和复杂性。虽然我们还不了解人类感觉运动协调的机制，但我们不可否认地同意，人类的运动技能是通过多年的实践和学习掌握的。PI已经建立了一个领先的项目，专注于应用先进的感官数据处理和基于模型的控制方法来增强机械机械手的功能。在此专业知识的基础上，拟议研究计划的主要目标是将最新的人工智能（AI）和机器学习技术与传统的基于模型的控制策略相结合，赋予机器人手臂和手类似人类的操作技能和适应性。具体而言，本研究旨在1)开发与操作任务相关的各种对象和特征的语义感知的多模态训练策略；2)研究逆强化学习（RL）策略，以了解人类专家目标导向操作任务背后的机制；3)整合机器人和环境动态模型的先验知识，以提高RL的性能。4)开发新的模拟训练环境以及有效学习的分层强化学习结构。机器学习在物理系统中的成功应用为我们设计和操作机电一体化系统的方式带来了巨大的突破潜力，使它们能够在复杂和不确定的环境中无缝运行，同时不断与人类和未知物体互动。通过该研究项目产生的新知识和基础技术的实用价值将对加拿大的制造业、物流、供应链和医疗保健行业等广泛的工业和公共部门产生重大影响。与该项目相关的研究任务横跨力学、机器人、控制理论、信号处理、人工智能和机器学习等不同学科，从而为我们的下一代高素质人才提供理想的培训环境。

The rise of collaborative robots promises a new era that robots can coexist with humans in our daily lives, be it home or workplace. Recent strides in machine learning have quickly transformed the way we design autonomous systems even under conditions previously considered too complex to replace humans (e.g. driving a car or playing the game of go). While these data-driven tools offer promises in knowledge-based reasoning such as analyzing image or audio data, it is still a question how such techniques can be transferred to the complex motor control of mechanical devices such as robotic arm and hand that are governed by principles of mechanics. Human extremities are vastly superior to artificial counterparts; this is not in the power and accuracy in executing the pre-programmed task repeatedly, but rather to the richness, adaptability and the sophistication of skilled movements for complex and unpredictable tasks. While we do not yet understand the mechanisms of human sensorimotor coordination, we agree undeniably that human motor skills are mastered through years of practice and learning. The PI has established a leading program focused on the application of advanced sensory data processing and model-based control approaches to enhancing the functionalities of mechanical manipulators. Building on this expertise, the main goal of the proposed research program is to combine recent AI (artificial intelligence) and machine learning techniques with traditional model-based control strategies to endow robotic arms and hands with human-like manipulation skills and adaptability. Specifically, this research aims 1) to develop multi-modal training strategies for semantic perception of various objects and features associated with manipulation tasks, 2) to investigate inverse reinforcement learning (RL) strategies to understand mechanisms behind goal-directed manipulation tasks by human experts, 3) to integrate a priori knowledge on dynamic models of the robot and the environment to enhance the performance of RL, and 4) to develop new simulated training environments as well as a hierarchical RL structure for efficient learning. Successful application of machine learning to physical systems holds enormous potential for breakthroughs in the way we design and operate mechatronic systems by allowing them to seamlessly operate within complex and uncertain environments while constantly interacting with humans and unknown objects. The practical value of the new knowledge and fundamental technologies to be generated through this research program will have a substantial impact on a wide range of industrial and public sectors including Canada's manufacturing, logistics, supply chain, and health care industries. Research tasks associated with this program cut across different disciplines of mechanics, robotics, control theory, signal processing, AI and machine learning, thereby providing ideal training environments to our next generation of highly qualified personnel.