CAREER: Manipulation of Novel Objects via Non-Smooth Implicit Learning

这个教师早期职业发展（CAREER）项目将创建物理启发的学习方法，用于数据高效的机器人操作新对象-也就是说，以前看不见的对象，机器人没有先验知识。其结果将使未来几代机器人能够在人类用户的日常生活中提供有意义的帮助。为了实现这一目标，机器人必须能够通过物理交互快速了解周围环境，特别是在精心控制的实验室条件之外的混乱环境中。这将要求机器人获得超越当前技术水平的新能力。该项目将为机器人提供确定周围物体的关键特征的能力-尽管形状，大小，颜色和材料不同-例如它们在触摸时是否移动，它们是软的还是硬的，以及它们是弯曲还是扭曲。机器人应该能够第一次进入一个房间，简单地调查该房间中的物体，然后安全地完成指定的任务。例如，一个家用机器人可能会在厨房里四处走动，遇到新的食物或烹饪工具，然后与这些物品互动，帮助准备一顿饭。该项目将推进一系列改善生活的机器人应用，包括家庭辅助护理，危险的搜索和救援行动或小规模制造。为了培养和激励下一代工程师，研究人员将与费城公立学区的教育工作者合作，开发一个利用机器人模拟器演示高中代数概念的教育单元。该项目汇集了非光滑动力学，学习和控制的概念，使机器人能够灵巧地操纵以前看不见的物体，使用从混乱的环境中真实的实时收集的数据。例如，机器人可以与一组新颖的物体进行至多几秒钟或几分钟的交互，然后以类似人类的灵巧性精确地执行复杂的任务，例如工具使用或手操作。这个项目的需要首先是由机器人和它的环境之间的相互作用，这是出了名的难以建模的功能机器人的依赖，第二，由基于模型和模拟到真实的控制方法的预测模型的依赖。本计画将所有的非平滑行为来源集合成一组接触力，以建立不连续接触驱动动力学的模型。 隐式损失函数本身使用凸优化来估计非光滑接触力，可以使用基于梯度的方法来最小化，以找到一组描述机器人-物体相互作用的物理学的光滑参数。为了提供机器人与世界互动的数据高效学习，该项目探索了以下三个主要研究方向：（1）开发具有物理启发结构的基础内隐学习框架，用于预测机器人世界的交互，（2）通过将触觉和视觉传感与运动预测相结合来动态操纵新的刚性和柔性物体，以及（3）将这两个学习系统与控制和强化学习相结合以获得闭环性能。总之，这些推力将提供新的机器人能力时，处理新的对象，在一系列的操纵任务，包括在手操纵和对象重新定位，全身操纵（使用四肢、躯干和其他身体部位）操纵、推动和拖动重物，和多个该奖项反映了NSF的法定使命，并被认为是值得通过使用基金会的学术价值和更广泛的影响审查标准。

This Faculty Early Career Development (CAREER) project will create physics-inspired learning methods for data-efficient robotic manipulation of novel objects – that is, previously unseen objects about which the robot has no prior knowledge. The result will enable future generations of robots to provide meaningful assistance throughout the daily lives of human users. To achieve this, robots must be able to quickly learn about their surroundings through physical interactions, particularly in chaotic settings beyond carefully controlled laboratory conditions. This will require robots to gain new capabilities beyond the current state of the art. This project will provide robots with the ability to determine critical characteristics of surrounding objects -- despite variations in shape, size, color, and material -- such as whether they move when touched, whether they are soft or stiff, and whether they bend or twist. A robot should be able to enter a room for the first time, briefly investigate the objects in that room, and then safely accomplish an assigned task. For example, an in-home robot might maneuver about a kitchen, encountering new food items or culinary tools, and then interact with those items to help prepare a meal. This project will advance a range of life-improving robotic applications, including in-home assistive care, dangerous search and rescue operations, or small-scale manufacturing. To train and inspire the next generation of engineers, investigators, working with educators in the Philadelphia Public School District, will develop an educational unit leveraging a robotic simulator to demonstrate concepts from high-school algebra.This project brings together concepts from non-smooth dynamics, learning, and control, to enable robots that perform dexterous manipulation of previously unseen objects, using data gathered in real time from a cluttered environment. For example, a robot may interact with a set of novel objects for at most a few seconds or minutes, then precisely perform, with human-like dexterity, complex tasks such as tool use or in-hand manipulation. The need for this project is driven first by the dependence of functional robotics on interaction between the robot and its environment, which is notoriously difficult to model, and second, by the reliance on predictive models of both model-based and sim-to-real methods for control. This project addresses the modeling of discontinuous contact-driven dynamics by gathering all sources of non-smooth behavior into a set of contact forces. An implicit loss function, which itself uses convex optimization to estimate non-smooth contact forces, can be minimized using gradient-based methods to find a set of the smooth parameters that describe the physics of robot-object interactions. To provide data-efficient learning of robot-world interaction, this project explores the following three primary research thrusts: (1) development of foundational implicit-learning frameworks with physics-inspired structure for predicting robot-world interactions, (2) dynamic manipulation of novel rigid and soft objects by unifying tactile and visual sensing with motion prediction, and (3) combining these two learning systems with control and reinforcement learning for closed-loop performance. Together, these thrusts will provide new robotic capabilities when dealing with novel objects, across a range of manipulation tasks including in-hand manipulation and object reorientation, whole-body manipulation (using limbs, torso, and other body parts) to maneuver, push, and drag heavy objects, and multi-arm manipulation of large or unwieldy objects.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.