CAREER: Self-supervised Representation Learning for Deformable Object Manipulation

可变形的对象;也就是说，那些随着压力而改变形状的物体，在我们的生活中无处不在。当我们叠衣服、给自己倒饮料、剥蔬菜时，人类一直在处理可变形的物体。尽管人类可以毫不费力地完成这些任务，但对于假设所有物体都是刚性的传统机器人技术来说，这是一个挑战。机器人可变形物体操作，如叠衣服或做饭，可以使那些不能自己轻松完成这些任务的人过上更独立的生活。这个教师早期职业发展（Career）项目将开发机器人学习操纵可变形物体的新方法。首先，机器人将学会预测它们的行为将如何导致物体变形；然后，机器人将使用这些预测来确定如何实现可变形物体操作任务。为了让公众更好地理解这项研究，研究人员将开发一个名为“Teach-a-Robot”的虚拟机器人界面，让世界上的任何人都能体验教机器人完成新任务的过程：用户上传自己完成任务的视频，然后观看机器人试图模仿他们。最后，研究人员将设计新的课程作业，展示机器人、计算机视觉和机器学习之间的相互作用。该项目的研究目标是探索机器人如何使用自监督表示学习来实现可变形物体的操作。给定一组机器人或人与可变形物体交互的演示，该项目将使机器人能够学习这些物体的表示，这些表示对操作有用。这项工作将探索如何使用自我监督学习来预测机器人对可变形物体的动作的影响；推断对象属性；识别大变形下的可变形物体部分；学习在同一类的可变形对象之间转移策略；学习从模拟到现实世界的视觉表现。第二个研究重点将探索机器人如何使用这些自监督表征来有效地学习策略来操纵可变形物体。这项工作将开发新的强化学习方法，可以执行多尺度推理，使机器人能够专注于与任务相关的低级对象细节以及全局对象配置。这项研究成果将为机器人与各种可变形物体互动打开新的大门，比如折叠布料、制作糕点的面团、在披萨面团上涂酱，以及剥蔬菜。该项目由跨部门机器人基础研究项目支持，由工程（ENG）和计算机与信息科学与工程（CISE）联合管理和资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Deformable objects; that is, those objects that change shape with pressure, are ubiquitous in our lives. Humans deal with deformable objects all the time when we fold laundry, pour ourselves a drink, and peel vegetables. Although such tasks are performed effortlessly by humans, they are challenging for traditional robotics techniques which assume that all objects are rigid. Robot deformable object manipulation, such as laundry folding or cooking, could enable those who cannot perform these tasks easily for themselves to live more independent lives. This Faculty Early Career Development (CAREER) project will develop novel methods for robots to learn to manipulate deformable objects. First, robots will learn to predict how their actions will cause an object to deform; robots will then use these predictions to determine how to achieve deformable object manipulation tasks. To enable the public to better understand this research, the investigators will develop a virtual robot interface called “Teach-a-Robot" that will enable anyone in the world to experience teaching a robot a new task: users will upload a video of themselves performing a task and then watch a robot attempting to imitate them. Finally, the investigators will design new course assignments that demonstrate the interplay between robotics, computer vision, and machine learning.The research objective of this project is to explore how self-supervised representation learning can be used by robots to achieve deformable object manipulation. Given a set of demonstrations of robots or people interacting with deformable objects, the project will enable robots to learn representations of these objects that are useful for manipulation. This effort will explore how self-supervised learning can be used to: predict the effect of a robot's actions on a deformable object; infer object properties; recognize deformable object parts under large deformations; learn to transfer policies between deformable objects of the same class; and learn visual representations that transfer well from simulation to the real world. The second research thrust will explore how robots can use these self-supervised representations to efficiently learn policies to manipulate deformable objects. This effort will develop novel methods for reinforcement learning that can perform multi-scale reasoning, enabling robots to focus on both task-relevant low-level object details as well as the global object configuration. This research effort will open new doors for robots to interact with a wide range of deformable objects, such as folding cloth, molding dough to make pastries, spreading sauce onto pizza dough, and peeling vegetables.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.