Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations
Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations
复制标题
DOI：
10.1109/iros55552.2023.10341728
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Yilun Hao;Ruinan Wang;Zhangjie Cao;Zihan Wang;Yuchen Cui;Dorsa Sadigh
中科院分区：
文献类型：
--
作者：
Yilun Hao;Ruinan Wang;Zhangjie Cao;Zihan Wang;Yuchen Cui;Dorsa Sadigh
多模式演示为机器人提供了丰富的信息，让他们了解世界。然而，当涉及到从人类演示中学习感觉运动控制政策时，这样的丰富可能并不总是导致良好的表现。无关的数据通道可能会导致状态过度规范，其中状态包含的通道不仅对决策毫无用处，而且可能会改变环境中的数据分布。状态过度规范会导致学习策略不能在训练数据分布之外推广等问题。在这项工作中，我们提出了掩蔽模仿学习(MIL)来解决状态过度指定问题，有选择地使用信息通道。具体地说，我们设计了一个带有二进制掩码的掩码策略网络来阻止某些模式。我们开发了一个双层优化算法，该算法学习这个掩码来准确地过滤过度指定的模态。实验证明，MIL算法在模拟域上的性能优于基线算法，并在真实机器人上采集的多模式数据集上有效地恢复了环境不变的模式。有关视频和补充细节，请访问：https://tinyurl.com/masked-il
Multimodal demonstrations provide robots with an abundance of information to make sense of the world. However, such abundance may not always lead to good performance when it comes to learning sensorimotor control policies from human demonstrations. Extraneous data modalities can lead to state over-specification, where the state contains modalities that are not only useless for decision-making but also can change data distribution across environments. State over-specification leads to issues such as the learned policy not generalizing outside of the training data distribution. In this work, we propose Masked Imitation Learning (MIL) to address state over-specification by selectively using informative modalities. Specifically, we design a masked policy network with a binary mask to block certain modalities. We develop a bi-level optimization algorithm that learns this mask to accurately filter over-specified modalities. We demonstrate empirically that MIL outperforms baseline algorithms in simulated domains and effectively recovers the environment-invariant modalities on a multimodal dataset collected on a real robot. Videos and supplemental details are at: https://tinyurl.com/masked-il