Adversarial Object Rearrangement in Constrained Environments with Heterogeneous Graph Neural Networks
Adversarial Object Rearrangement in Constrained Environments with Heterogeneous Graph Neural Networks
复制标题
DOI：
10.1109/iros55552.2023.10342412
复制
发表时间：
2023-09
期刊：
影响因子：
--
通讯作者：
Xibai Lou;Houjian Yu;Ross Worobel;Yang Yang-Yang;Changhyun Choi
中科院分区：
文献类型：
--
作者：
Xibai Lou;Houjian Yu;Ross Worobel;Yang Yang-Yang;Changhyun Choi
真实的世界中的对抗性对象重排（例如，以前看不见的或厨房和商店中的超大物品）可以从理解任务场景中受益，任务场景固有地需要诸如当前对象、目标对象和环境约束之类的异构组件。这些组件之间的语义关系彼此不同，对于多技能机器人在日常场景中有效执行至关重要。我们提出了一个分层的机器人操作系统，学习底层的关系，并最大限度地提高其不同技能的协作能力（例如，PICK-PLACE，PUSH）用于在受限环境中重新排列对抗对象。高级协调器采用异构图神经网络（HetGNN），其对当前对象、目标对象和环境约束进行推理;低级基于3D卷积神经网络的演员执行动作原语。我们的方法完全在模拟中训练，在真实世界的实验中，平均成功率为87.88%，规划成本为12.82，超过了所有基线方法。补充材料可在https://sites.google.com/umn.edu/versatile-rearrangement上查阅。
Adversarial object rearrangement in the real world (e.g., previously unseen or oversized items in kitchens and stores) could benefit from understanding task scenes, which inherently entail heterogeneous components such as current objects, goal objects, and environmental constraints. The semantic relationships among these components are distinct from each other and crucial for multi-skilled robots to perform efficiently in everyday scenarios. We propose a hierarchical robotic manipulation system that learns the underlying relationships and maximizes the collaborative power of its diverse skills (e.g., PICK-PLACE, PUSH) for rearranging adversarial objects in constrained environments. The high-level coordinator employs a heterogeneous graph neural network (HetGNN), which reasons about the current objects, goal objects, and environmental constraints; the low-level 3D Convolutional Neural Network-based actors execute the action primitives. Our approach is trained entirely in simulation, and achieved an average success rate of 87.88% and a planning cost of 12.82 in real-world experiments, surpassing all baseline methods. Supplementary material is available at https://sites.google.com/umn.edu/versatile-rearrangement.