FRR: Semi-Structured, Under-Specified, Partially-Observable Robotic Rearrangement

该项目旨在开发先进的技术，使智能机器人能够高效、自主地与家庭和杂货店等日常人类环境中的物体互动，给出通用的、自然语言的任务描述。这项技术解决了重大的社会问题，包括支持老年人独立生活。随着人们年龄的增长，由于视力受损、家庭危险和虚弱，行动不便往往会导致频繁和严重的伤害。家用机器人可以帮助完成取回、转移和重新排列物品的任务，比如摆放餐桌或从橱柜后面抓起一个罐子。同样，重新安排机器人可以帮助执行零售运营中的劳动密集型、重复性库存管理任务。像整理货架和重新进货这样的任务是劳动密集型的，可能会导致受伤，而这些工作往往很难填补，并且流失率很高。在人类的半结构化环境中可靠地执行这些对象操纵任务具有很大的不确定性，对现代机器人来说仍然是具有挑战性的。此外，在现代家庭或杂货店等半结构化环境中，新对象经常被引入和操纵，这进一步使机器人的任务变得复杂。特别是，在这些场景中，自主机器人在解决操纵任务时面临着多个障碍，包括(1)机器人必须从非专家人类给出的隐含任务规范中推导出完整的操纵计划，(2)机器人必须在对象的先验知识并不总是可用的环境中实现准确的场景理解，以及(3)规划过程必须遵守现实的部分可观察性约束，在这种情况下，RGB-D摄像机等传感器一次只能检测场景的一部分。为了解决最先进技术的局限性，该项目将开发一个新的迭代场景理解和重新安排规划框架。该框架将逐步建立机器人环境的越来越准确的模型。自适应场景表示将包含部分观察到的对象的身份、几何和可能位置，达到足以安全和有效地解决人类分配的任务的水平。这种表示将被用来在现实可见性约束下有效地执行由人提供的作为自然语言命令的操纵任务。该项目还将为该框架的高效实施奠定基础，旨在提供自然、高质量的解决方案，以实现理想的保证，如安全性、解决方案的完整性和解决方案的最优化。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The project aims to develop advanced technologies for intelligent robots to efficiently and autonomously interact with objects in everyday, human environments, such as homes and grocery stores, given general, natural language task descriptions. This technology addresses significant societal issues, including the support of older adults in independent living. As people age, reduced mobility often leads to frequent and severe injuries due to impaired vision, home hazards, and weakness. Household robots can assist with tasks like retrieving, transferring, and rearranging items, such as setting up a dinner table or grabbing a jar from the back of a cabinet. Similarly, rearranging robots can assist with labor-intensive, repetitive inventory management tasks in retail operations. Such tasks, like tidying and restocking shelves, are labor-intensive and can lead to injuries, while these jobs are often difficult to fill and have high turnover rates.Reliably performing these object manipulation tasks in human, semi-structured environments involves significant uncertainty and remains challenging for modern robotics. Furthermore, new objects are frequently introduced and manipulated in semi-structured environments, such as modern homes or grocery stores, further complicating the task for robots. In particular, autonomous robots face multiple hurdles in solving manipulation tasks in these scenarios, including (1) a robot must derive a complete manipulation plan from implicit task specifications given by non-expert humans, (2) the robot must achieve accurate scene understanding in environments where prior knowledge of objects is not always available, and (3) the planning process must respect realistic partial observability constraints, where sensors like RGB-D cameras can only inspect portions of a scene at a time. To address the limitations of the state-of-the-art, the project will develop a novel Iterative Scene Understanding and Rearrangement Planning framework. The framework will build increasingly accurate models of a robot's environment progressively. The adaptive scene representation will contain the identities, geometries, and possible locations of partially observed objects, to a level sufficient for safely and effectively resolving human-assigned tasks. This representation will be leveraged to efficiently execute manipulation tasks provided by people as natural language commands under realistic visibility constraints. The project will also lay the groundwork for efficient implementations of this framework, aiming to deliver natural, high-quality solutions that achieve desirable guarantees, such as safety, resolution completeness, and solution optimality.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.