FRR: Semi-Structured, Under-Specified, Partially-Observable Robotic Rearrangement

该项目旨在开发先进的技术，使智能机器人能够有效地自主地与日常人类环境中的物体进行交互，例如家庭和杂货店，给出一般的自然语言任务描述。这项技术解决了重大的社会问题，包括支持老年人独立生活。随着人们年龄的增长，由于视力受损、家庭危险和虚弱，行动能力下降往往会导致频繁和严重的伤害。家用机器人可以协助完成检索，转移和重新排列物品等任务，例如设置餐桌或从橱柜后面抓取罐子。同样，重新安排机器人可以协助零售业务中的劳动密集型，重复性库存管理任务。这些任务，如整理和补充货架，是劳动密集型的，可能会导致伤害，而这些工作往往很难填补，并具有很高的流动率。在人类，半结构化的环境中可靠地执行这些对象操作任务涉及很大的不确定性，对现代机器人来说仍然具有挑战性。此外，在半结构化的环境中，如现代家庭或杂货店，经常会引入和操纵新的物体，这使得机器人的任务更加复杂。特别地，自主机器人在解决这些场景中的操纵任务时面临多个障碍，包括（1）机器人必须从非专家人类给出的隐式任务规范导出完整的操纵计划，（2）机器人必须在对象的先验知识并不总是可用的环境中实现准确的场景理解，以及（3）规划过程必须尊重现实的部分可观测性约束，像RGB-D相机这样的传感器一次只能检查场景的一部分。为了解决现有技术的局限性，该项目将开发一个新的迭代场景理解和重排规划框架。该框架将逐步构建越来越准确的机器人环境模型。 自适应场景表示将包含部分观察到的对象的身份，几何形状和可能的位置，达到足以安全有效地解决人类分配任务的水平。这种表示将被利用来有效地执行由人提供的操作任务，作为现实可见性约束下的自然语言命令。该项目还将为有效实施该框架奠定基础，旨在提供自然，高质量的解决方案，实现理想的保证，如安全性，解决方案的完整性和解决方案的最优性。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

The project aims to develop advanced technologies for intelligent robots to efficiently and autonomously interact with objects in everyday, human environments, such as homes and grocery stores, given general, natural language task descriptions. This technology addresses significant societal issues, including the support of older adults in independent living. As people age, reduced mobility often leads to frequent and severe injuries due to impaired vision, home hazards, and weakness. Household robots can assist with tasks like retrieving, transferring, and rearranging items, such as setting up a dinner table or grabbing a jar from the back of a cabinet. Similarly, rearranging robots can assist with labor-intensive, repetitive inventory management tasks in retail operations. Such tasks, like tidying and restocking shelves, are labor-intensive and can lead to injuries, while these jobs are often difficult to fill and have high turnover rates.Reliably performing these object manipulation tasks in human, semi-structured environments involves significant uncertainty and remains challenging for modern robotics. Furthermore, new objects are frequently introduced and manipulated in semi-structured environments, such as modern homes or grocery stores, further complicating the task for robots. In particular, autonomous robots face multiple hurdles in solving manipulation tasks in these scenarios, including (1) a robot must derive a complete manipulation plan from implicit task specifications given by non-expert humans, (2) the robot must achieve accurate scene understanding in environments where prior knowledge of objects is not always available, and (3) the planning process must respect realistic partial observability constraints, where sensors like RGB-D cameras can only inspect portions of a scene at a time. To address the limitations of the state-of-the-art, the project will develop a novel Iterative Scene Understanding and Rearrangement Planning framework. The framework will build increasingly accurate models of a robot's environment progressively. The adaptive scene representation will contain the identities, geometries, and possible locations of partially observed objects, to a level sufficient for safely and effectively resolving human-assigned tasks. This representation will be leveraged to efficiently execute manipulation tasks provided by people as natural language commands under realistic visibility constraints. The project will also lay the groundwork for efficient implementations of this framework, aiming to deliver natural, high-quality solutions that achieve desirable guarantees, such as safety, resolution completeness, and solution optimality.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.