The VoxWorld Platform for Multimodal Embodied Agents
The VoxWorld Platform for Multimodal Embodied Agents
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Nikhil Krishnaswamy;W. Pickard;Brittany E. Cates;Nathaniel Blanchard;J. Pustejovsky
中科院分区：
文献类型：
--
作者：
Nikhil Krishnaswamy;W. Pickard;Brittany E. Cates;Nathaniel Blanchard;J. Pustejovsky
我们提出了一个五年的回顾VoxWorld平台的发展，首先介绍了作为一个多模态平台建模的运动语言，已发展成为一个平台，用于快速构建和部署具体的代理与上下文和态势感知，能够与人类在多种形式的互动，并探索他们的环境。特别是，我们讨论了从VoxML建模语言的理论基础到一个平台的演变，该平台可容纳神经和符号输入，以构建能够进行多模态交互和混合推理的代理。我们专注于三个不同的代理实现和所需的功能，以适应所有这些：戴安娜，一个虚拟的协作代理; Kirby，一个移动的机器人;和BabyBAW，代理谁自我引导自己的探索世界。
We present a five-year retrospective on the development of the VoxWorld platform, first introduced as a multimodal platform for modeling motion language, that has evolved into a platform for rapidly building and deploying embodied agents with contextual and situational awareness, capable of interacting with humans in multiple modalities, and exploring their environments. In particular, we discuss the evolution from the theoretical underpinnings of the VoxML modeling language to a platform that accommodates both neural and symbolic inputs to build agents capable of multimodal interaction and hybrid reasoning. We focus on three distinct agent implementations and the functionality needed to accommodate all of them: Diana, a virtual collaborative agent; Kirby, a mobile robot; and BabyBAW, an agent who self-guides its own exploration of the world.