CAREER: Enhancing Robot Physical Intelligence via Crowdsourced Surrogate Learning

该学院早期职业发展（CAREER）资助开发了一种新的方法，以提高人类和机器人手对物体的熟练操作。与灵巧物体操作相关的挑战一直是限制机器人在许多生产和服务应用中应用的瓶颈。该项目介绍了一种新型的筷子状机器人和相关方法，该方法将允许机器人通过向分布在物理位置和时间上的一大群人学习来逐渐发展“物理智能”（可以转移到新情况下的技能）。该项目使用“众包”从使用筷子机器人的人那里获得大量数据，以指导类似的远程机器人进行工业和社会活动。在遥操作过程中观察到的感觉运动控制信号解析的一种新的人工智能控制器，以提高自主机器人的技能。这个控制器的一个有趣的功能是它能够从以前观察到的技能组成新的对象操作技能。该项目开发了一种用户训练方法，利用机器人获得的物理智能来主动引导和提高人类的机器人遥操作技能。这些创新有望通过提高机器人和人类操作技能在各种应用中的灵活性来促进国民健康，繁荣和福利，包括制造，医疗手术和家庭援助。该项目包括一个教育部分，吸引来自不同背景的大学预科和大学年龄的学生。该项目将开发和测试一个新的重点，提高人类和机器人手执行的灵巧物体操作技能。该项目提出了三个新颖的想法。 首先，PI将使用Amazon Mechanical Turk实现一个远程操作系统（用户界面、用户管理系统和机器人学习数据库），将一组人类操作员与远程机器人连接起来。这种方法被称为“众包代理学习”（CSL），它建立了一个传感器、电机和机器视觉数据的数据库，而大量的人分别使用一种新型的筷子状机器人来远程操作类似的机器人来执行工业和社会任务。这些人类受试者实验将提供后续机器人技能学习和推广所需的数据，使用一种称为“机器人复合学习”（RCL）的方法。RCL实现了一个离散化的关节状态空间的机器人和物体被抓住/操纵，沿着的统计推断技术，组成新的命令信号-实时-使两指机器人手学习从未展示过的技能，如新的，在手的小物品的操作。最后，PI将开发一种“互惠技能诱导”方法，使CSL/RCL系统能够通过使用“虚拟夹具”提供触觉引导来主动引导和提高人类的机器人遥操作技能，该虚拟夹具会随着新技能的学习而消失。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估而被认为值得支持。

This Faculty Early Career Development (CAREER) grant develops a novel way to enhance skilled manipulation of objects as performed by human and robotic hands. Challenges associated with dexterous object manipulation have been a bottleneck limiting the application of robots in many production and service applications. This project introduces a novel chopsticks-like robot and associated methods that will allow robots to gradually develop "physical intelligence" (skills that can be transferred to novel situations) by learning from a large group of people, who are distributed across physical location and time. The project uses "crowdsourcing" to obtain a large set of data from people using the chopsticks robot to guide a similar, remote robot through industrial and social activities. Sensorimotor control signals observed during teleoperation are parsed by a novel artificial intelligence controller to improve autonomous robot skills. An interesting feature of this controller is its ability to compose new object manipulation skills from parts of previously observed skills. The project develops a user training method that leverages the robot's acquired physical intelligence to actively guide and improve robot teleoperation skills of humans. The innovations promise to advance the national health, prosperity and welfare by improving dexterity of robotic and human manipulation skills in a variety of applications, including manufacturing, medical surgery, and home assistance. The project includes an educational component that engages pre-college and college-age students from diverse backgrounds.This project will develop and test a novel focus on improving dexterous object manipulation skills performed by human and robotic hands. The project advances three novel ideas. First, the PI will use Amazon Mechanical Turk to implement a teleoperation system (user interface, user management system, and robot learning database) linking a group of human operators with remote robots. This approach, called "Crowdsourced Surrogate Learning" (CSL), builds a database of sensor, motor, and machine vision data collected while a large number of people separately use a novel chopsticks-like robot to teleoperate a similar robot to conduct industrial and social tasks. These human subjects experiments will provide the data needed for subsequent robot skill learning and generalization using an approach called "Robot Composite Learning" (RCL). RCL implements a discretized joint state space of the robot and the object being grasped/manipulated, along with a statistical inference technique, to compose new command signals - in real-time - enabling the two-fingered robot hand to learn never-demonstrated skills such as new, in-hand manipulations of small items. Finally, the PI will develop a "Reciprocal Skill Induction" method that enables the CSL/RCL system to actively guide and improve the robot teleoperation skills of humans by providing haptic guidance using "virtual fixtures" that fade as a novel skill is learned.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.