CAREER: Self-supervised Representation Learning for Deformable Object Manipulation

可变形物体，即那些随着压力而改变形状的物体，在我们的生活中无处不在。当我们叠衣服、倒饮料、剥蔬菜时，人类总是要与可变形的物体打交道。尽管这样的任务由人类毫不费力地完成，但它们对传统的机器人技术构成了挑战，传统的机器人技术假设所有物体都是刚性的。机器人对可变形物体的操纵，如叠衣服或做饭，可以让那些不能轻松完成这些任务的人过上更独立的生活。这个学院早期职业发展(Career)项目将开发新的方法，让机器人学习操纵可变形的物体。首先，机器人将学会预测它们的动作将如何导致物体变形；然后，机器人将使用这些预测来确定如何实现可变形的物体操纵任务。为了让公众更好地了解这项研究，研究人员将开发一种名为“Teach-a-Robot”的虚拟机器人界面，使世界上任何人都能体验到教授机器人一项新任务：用户将上传自己执行任务的视频，然后观看机器人试图模仿他们。最后，研究人员将设计新的课程作业，展示机器人学、计算机视觉和机器学习之间的相互作用。本项目的研究目标是探索机器人如何使用自我监督的表征学习来实现可变形对象的操作。给出一组机器人或人与可变形物体互动的演示，该项目将使机器人能够学习这些物体的表示，这些表示对操纵有用。这项工作将探索如何使用自我监督学习来：预测机器人的动作对可变形对象的影响；推断对象属性；识别大变形下的可变形对象部分；学习在同一类可变形对象之间传输策略；以及学习从模拟到真实世界的良好转移的视觉表示。第二个研究重点将探索机器人如何使用这些自我监督的表示来有效地学习策略来操作可变形对象。这项工作将开发新的强化学习方法，可以执行多尺度推理，使机器人能够专注于与任务相关的低层对象细节以及全局对象配置。这项研究工作将为机器人与各种各样的可变形物体互动打开新的大门，如折叠布、成型面团制作糕点、在披萨面团上涂抹酱汁和剥蔬菜。该项目得到了机器人基础研究项目的支持，该项目由工程总监(ENG)和计算机和信息科学与工程(CEISE)共同管理和资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Deformable objects; that is, those objects that change shape with pressure, are ubiquitous in our lives. Humans deal with deformable objects all the time when we fold laundry, pour ourselves a drink, and peel vegetables. Although such tasks are performed effortlessly by humans, they are challenging for traditional robotics techniques which assume that all objects are rigid. Robot deformable object manipulation, such as laundry folding or cooking, could enable those who cannot perform these tasks easily for themselves to live more independent lives. This Faculty Early Career Development (CAREER) project will develop novel methods for robots to learn to manipulate deformable objects. First, robots will learn to predict how their actions will cause an object to deform; robots will then use these predictions to determine how to achieve deformable object manipulation tasks. To enable the public to better understand this research, the investigators will develop a virtual robot interface called “Teach-a-Robot" that will enable anyone in the world to experience teaching a robot a new task: users will upload a video of themselves performing a task and then watch a robot attempting to imitate them. Finally, the investigators will design new course assignments that demonstrate the interplay between robotics, computer vision, and machine learning.The research objective of this project is to explore how self-supervised representation learning can be used by robots to achieve deformable object manipulation. Given a set of demonstrations of robots or people interacting with deformable objects, the project will enable robots to learn representations of these objects that are useful for manipulation. This effort will explore how self-supervised learning can be used to: predict the effect of a robot's actions on a deformable object; infer object properties; recognize deformable object parts under large deformations; learn to transfer policies between deformable objects of the same class; and learn visual representations that transfer well from simulation to the real world. The second research thrust will explore how robots can use these self-supervised representations to efficiently learn policies to manipulate deformable objects. This effort will develop novel methods for reinforcement learning that can perform multi-scale reasoning, enabling robots to focus on both task-relevant low-level object details as well as the global object configuration. This research effort will open new doors for robots to interact with a wide range of deformable objects, such as folding cloth, molding dough to make pastries, spreading sauce onto pizza dough, and peeling vegetables.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.