FRR: Semi-Structured, Under-Specified, Partially-Observable Robotic Rearrangement

该项目旨在为智能机器人开发先进技术，使其能够有效自主地与日常人类环境中的物体进行交互，例如家庭和杂货店，给出一般的自然语言任务描述。这项技术解决了重大的社会问题，包括支持老年人独立生活。随着人们年龄的增长，由于视力受损、家庭危险和身体虚弱，行动能力降低往往会导致频繁和严重的伤害。家用机器人可以协助完成取回、转移和重新排列物品等任务，比如摆放餐桌或从橱柜后面抓起罐子。同样，重新安排机器人可以帮助完成零售业务中劳动密集型、重复性的库存管理任务。这些工作，如整理和补充货架，是劳动密集型的，可能会导致受伤，而这些工作往往很难填补，而且流动率很高。在人类半结构化环境中可靠地执行这些对象操作任务涉及很大的不确定性，对现代机器人技术来说仍然是一个挑战。此外，在半结构化的环境中，如现代家庭或杂货店，新物体经常被引入和操纵，这进一步使机器人的任务复杂化。特别是，自主机器人在解决这些场景中的操作任务时面临多重障碍，包括(1)机器人必须从非专家给出的隐式任务规范中推导出完整的操作计划，(2)机器人必须在并不总是可用的物体先验知识的环境中实现准确的场景理解，以及(3)规划过程必须尊重现实的部分可观察性约束。像RGB-D相机这样的传感器一次只能检查场景的一部分。为了解决最先进技术的局限性，该项目将开发一种新颖的迭代场景理解和重排规划框架。该框架将逐步建立越来越精确的机器人环境模型。自适应场景表示将包含部分观察对象的身份、几何形状和可能的位置，达到足以安全有效地解决人工分配任务的水平。这种表示将被用来在现实可见性约束下有效地执行人们作为自然语言命令提供的操作任务。该项目还将为该框架的有效实现奠定基础，旨在交付自然的、高质量的解决方案，以实现理想的保证，例如安全性、解决方案完整性和解决方案最优性。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The project aims to develop advanced technologies for intelligent robots to efficiently and autonomously interact with objects in everyday, human environments, such as homes and grocery stores, given general, natural language task descriptions. This technology addresses significant societal issues, including the support of older adults in independent living. As people age, reduced mobility often leads to frequent and severe injuries due to impaired vision, home hazards, and weakness. Household robots can assist with tasks like retrieving, transferring, and rearranging items, such as setting up a dinner table or grabbing a jar from the back of a cabinet. Similarly, rearranging robots can assist with labor-intensive, repetitive inventory management tasks in retail operations. Such tasks, like tidying and restocking shelves, are labor-intensive and can lead to injuries, while these jobs are often difficult to fill and have high turnover rates.Reliably performing these object manipulation tasks in human, semi-structured environments involves significant uncertainty and remains challenging for modern robotics. Furthermore, new objects are frequently introduced and manipulated in semi-structured environments, such as modern homes or grocery stores, further complicating the task for robots. In particular, autonomous robots face multiple hurdles in solving manipulation tasks in these scenarios, including (1) a robot must derive a complete manipulation plan from implicit task specifications given by non-expert humans, (2) the robot must achieve accurate scene understanding in environments where prior knowledge of objects is not always available, and (3) the planning process must respect realistic partial observability constraints, where sensors like RGB-D cameras can only inspect portions of a scene at a time. To address the limitations of the state-of-the-art, the project will develop a novel Iterative Scene Understanding and Rearrangement Planning framework. The framework will build increasingly accurate models of a robot's environment progressively. The adaptive scene representation will contain the identities, geometries, and possible locations of partially observed objects, to a level sufficient for safely and effectively resolving human-assigned tasks. This representation will be leveraged to efficiently execute manipulation tasks provided by people as natural language commands under realistic visibility constraints. The project will also lay the groundwork for efficient implementations of this framework, aiming to deliver natural, high-quality solutions that achieve desirable guarantees, such as safety, resolution completeness, and solution optimality.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.