TidyBot: Personalized Robot Assistance with Large Language Models
TidyBot: Personalized Robot Assistance with Large Language Models
复制标题
DOI：
10.1007/s10514-023-10139-z
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
Jimmy Wu;Rika Antonova;Adam Kan;Marion Lepert;Andy Zeng;Shuran Song;J. Bohg;S. Rusinkiewicz;T. Funkhouser
中科院分区：
文献类型：
--
作者：
Jimmy Wu;Rika Antonova;Adam Kan;Marion Lepert;Andy Zeng;Shuran Song;J. Bohg;S. Rusinkiewicz;T. Funkhouser
为了让机器人有效地个性化物理帮助，它必须学习用户的偏好，这些偏好通常可以重新应用于未来的场景。在这项工作中，我们研究了家庭清洁的个性化与机器人，可以整理房间，拿起物体，把它们放在一边。一个关键的挑战是确定放置每个对象的适当位置，因为人们的偏好可能会因个人品味或文化背景而有很大差异。例如，一个人可能更喜欢将衬衫存放在抽屉里，而另一个人可能更喜欢将它们放在架子上。我们的目标是建立一个系统，通过与特定的人进行先前的互动，可以从少数几个例子中学习这种偏好。我们表明，机器人可以结合联合收割机基于语言的规划和感知与大型语言模型的少数镜头摘要功能，推断广义用户的喜好，广泛适用于未来的互动。这种方法可以快速适应，并在我们的基准数据集中对看不见的对象实现了91.2%的准确率。我们还展示了我们的方法在现实世界中的移动的机械手TidyBot，成功地把85.0%的对象在现实世界中的测试场景。
For a robot to personalize physical assistance effectively, it must learn user preferences that can be generally reapplied to future scenarios. In this work, we investigate personalization of household cleanup with robots that can tidy up rooms by picking up objects and putting them away. A key challenge is determining the proper place to put each object, as people’s preferences can vary greatly depending on personal taste or cultural background. For instance, one person may prefer storing shirts in the drawer, while another may prefer them on the shelf. We aim to build systems that can learn such preferences from just a handful of examples via prior interactions with a particular person. We show that robots can combine language-based planning and perception with the few-shot summarization capabilities of large language models to infer generalized user preferences that are broadly applicable to future interactions. This approach enables fast adaptation and achieves 91.2% accuracy on unseen objects in our benchmark dataset. We also demonstrate our approach on a real-world mobile manipulator called TidyBot, which successfully puts away 85.0% of objects in real-world test scenarios.