Integrated Machine Learning and Control for Synthesis of Dexterous Manipulation Skills

协作机器人的兴起预示着一个新的时代，机器人可以在我们的日常生活中与人类共存，无论是在家里还是在工作场所。机器学习的最新进展迅速改变了我们设计自主系统的方式，即使在以前被认为过于复杂而无法取代人类的情况下（例如驾驶汽车或玩围棋）。虽然这些数据驱动工具在基于知识的推理（如分析图像或音频数据）中提供了希望，但如何将这些技术转移到机械设备（如机械臂和机械手）的复杂电机控制中仍然是一个问题。人类的四肢比人造的四肢有着上级的优势;这并不在于重复执行预先编程的任务的能力和准确性，而是在于复杂和不可预测的任务的丰富性、适应性和熟练动作的复杂性。虽然我们还不了解人类感觉运动协调的机制，但我们一致认为，人类的运动技能是通过多年的实践和学习掌握的。PI已经建立了一个领先的计划，专注于应用先进的传感数据处理和基于模型的控制方法来增强机械操作器的功能。在此专业知识的基础上，拟议研究计划的主要目标是将联合收割机最新的AI（人工智能）和机器学习技术与传统的基于模型的控制策略相结合，以赋予机器人手臂和手类似人类的操作技能和适应性。具体来说，本研究的目的是：1）开发多模态训练策略，用于与操作任务相关的各种对象和特征的语义感知，2）研究反向强化学习（RL）策略，以理解人类专家目标导向操作任务背后的机制，3）整合机器人和环境动态模型的先验知识，以提高RL的性能，以及4）开发新的模拟训练环境以及用于有效学习的分层RL结构。机器学习在物理系统中的成功应用为我们设计和操作机电一体化系统的方式带来了巨大的突破潜力，使它们能够在复杂和不确定的环境中无缝操作，同时不断与人类和未知物体进行交互。通过这项研究计划产生的新知识和基础技术的实用价值将对包括加拿大制造业，物流，供应链和医疗保健行业在内的广泛的工业和公共部门产生重大影响。与该计划相关的研究任务跨越了机械，机器人，控制理论，信号处理，人工智能和机器学习的不同学科，从而为我们的下一代高素质人才提供理想的培训环境。

The rise of collaborative robots promises a new era that robots can coexist with humans in our daily lives, be it home or workplace. Recent strides in machine learning have quickly transformed the way we design autonomous systems even under conditions previously considered too complex to replace humans (e.g. driving a car or playing the game of go). While these data-driven tools offer promises in knowledge-based reasoning such as analyzing image or audio data, it is still a question how such techniques can be transferred to the complex motor control of mechanical devices such as robotic arm and hand that are governed by principles of mechanics. Human extremities are vastly superior to artificial counterparts; this is not in the power and accuracy in executing the pre-programmed task repeatedly, but rather to the richness, adaptability and the sophistication of skilled movements for complex and unpredictable tasks. While we do not yet understand the mechanisms of human sensorimotor coordination, we agree undeniably that human motor skills are mastered through years of practice and learning. The PI has established a leading program focused on the application of advanced sensory data processing and model-based control approaches to enhancing the functionalities of mechanical manipulators. Building on this expertise, the main goal of the proposed research program is to combine recent AI (artificial intelligence) and machine learning techniques with traditional model-based control strategies to endow robotic arms and hands with human-like manipulation skills and adaptability. Specifically, this research aims 1) to develop multi-modal training strategies for semantic perception of various objects and features associated with manipulation tasks, 2) to investigate inverse reinforcement learning (RL) strategies to understand mechanisms behind goal-directed manipulation tasks by human experts, 3) to integrate a priori knowledge on dynamic models of the robot and the environment to enhance the performance of RL, and 4) to develop new simulated training environments as well as a hierarchical RL structure for efficient learning. Successful application of machine learning to physical systems holds enormous potential for breakthroughs in the way we design and operate mechatronic systems by allowing them to seamlessly operate within complex and uncertain environments while constantly interacting with humans and unknown objects. The practical value of the new knowledge and fundamental technologies to be generated through this research program will have a substantial impact on a wide range of industrial and public sectors including Canada's manufacturing, logistics, supply chain, and health care industries. Research tasks associated with this program cut across different disciplines of mechanics, robotics, control theory, signal processing, AI and machine learning, thereby providing ideal training environments to our next generation of highly qualified personnel.