DANLI: Deliberative Agent for Following Natural Language Instructions
DANLI: Deliberative Agent for Following Natural Language Instructions
复制标题
DOI：
10.48550/arxiv.2210.12485
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Yichi Zhang;Jianing Yang;Jiayi Pan;Shane Storks;N. Devraj;Ziqiao Ma;Keunwoo Peter Yu;Yuwei Bao;J. Chai
中科院分区：
文献类型：
--
作者：
Yichi Zhang;Jianing Yang;Jiayi Pan;Shane Storks;N. Devraj;Ziqiao Ma;Keunwoo Peter Yu;Yuwei Bao;J. Chai
近年来，嵌入人工智能代理的工作越来越多，这些代理可以通过遵循人类语言指令来执行任务。然而，大多数智能体都是被动的，这意味着它们只是学习和模仿训练数据中遇到的行为。这些反应剂不足以应付长期复杂的任务。为了解决这一限制，我们提出了一种神经符号审议代理，它在遵循语言指令的同时，根据从过去经验中获得的神经和符号表征（例如，自然语言和自我中心视觉），主动应用推理和规划。我们表明，在具有挑战性的TEACh基准测试中，我们的审议代理比反应基准实现了超过70%的改进。此外，潜在的推理和规划过程，以及我们的模块化框架，为智能体的行为提供了令人印象深刻的透明度和可解释性。这使我们能够深入了解智能体的能力，从而为未来的指令遵循具体化智能体带来挑战和机遇。代码可在https://github.com/sled-group/DANLI上获得。
Recent years have seen an increasing amount of work on embodied AI agents that can perform tasks by following human language instructions. However, most of these agents are reactive, meaning that they simply learn and imitate behaviors encountered in the training data. These reactive agents are insufficient for long-horizon complex tasks. To address this limitation, we propose a neuro-symbolic deliberative agent that, while following language instructions, proactively applies reasoning and planning based on its neural and symbolic representations acquired from past experience (e.g., natural language and egocentric vision). We show that our deliberative agent achieves greater than 70% improvement over reactive baselines on the challenging TEACh benchmark. Moreover, the underlying reasoning and planning processes, together with our modular framework, offer impressive transparency and explainability to the behaviors of the agent. This enables an in-depth understanding of the agent’s capabilities, which shed light on challenges and opportunities for future embodied agents for instruction following. The code is available at https://github.com/sled-group/DANLI.