REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction
REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction
复制标题
DOI：
10.48550/arxiv.2306.15724
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Zeyi Liu;Arpit Bahety;Shuran Song
中科院分区：
文献类型：
--
作者：
Zeyi Liu;Arpit Bahety;Shuran Song
自动检测和分析失败执行的能力对于可解释和强大的机器人系统至关重要。最近，大型语言模型（LLM）已经表现出强大的推理能力的文本输入。为了利用LLM的机器人故障解释的力量，我们引入了REFLECT，一个框架，查询LLM故障推理的基础上，从多传感器观测产生的机器人过去的经验的分层总结。失败解释可以进一步指导基于语言的计划者纠正失败并完成任务。为了系统地评估框架，我们创建了RoboFail数据集，其中包含各种任务和失败场景。我们证明，基于LLM的框架是能够生成信息丰富的失败的解释，帮助成功的纠正计划。
The ability to detect and analyze failed executions automatically is crucial for an explainable and robust robotic system. Recently, Large Language Models (LLMs) have demonstrated strong reasoning abilities on textual inputs. To leverage the power of LLMs for robot failure explanation, we introduce REFLECT, a framework which queries LLM for failure reasoning based on a hierarchical summary of robot past experiences generated from multisensory observations. The failure explanation can further guide a language-based planner to correct the failure and complete the task. To systematically evaluate the framework, we create the RoboFail dataset with a variety of tasks and failure scenarios. We demonstrate that the LLM-based framework is able to generate informative failure explanations that assist successful correction planning.