Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors
Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors
复制标题
DOI：
10.1109/icra48506.2021.9561847
复制
发表时间：
2020-12
期刊：
影响因子：
--
通讯作者：
Michelle A. Lee;Matthew Tan;Yuke Zhu;J. Bohg
中科院分区：
文献类型：
--
作者：
Michelle A. Lee;Matthew Tan;Yuke Zhu;J. Bohg
使用来自多个模态的传感器数据提供了对冗余和互补特征进行编码的机会，当一个模态被破坏或有噪声时，这些特征可能是有用的。人类每天都在这样做，在视觉挑战性的环境中依靠触觉和本体感受反馈。然而，机器人可能并不总是知道它们的传感器何时损坏，因为即使损坏的传感器也可以返回有效值。在这项工作中，我们介绍了跨模态补偿模型（CCM），它可以检测损坏的传感器模态，并对它们进行补偿。CMM是一种通过自我监督学习的表示模型，它利用单峰重建损失进行腐败检测。CCM然后丢弃损坏的模态，并使用来自剩余传感器的信息对其进行补偿。我们表明，CCM学习了丰富的状态表示，可用于通过强化学习学习的操纵策略，即使在策略推出期间输入模态以训练期间未看到的方式被损坏。
Using sensor data from multiple modalities presents an opportunity to encode redundant and complementary features that can be useful when one modality is corrupted or noisy. Humans do this everyday, relying on touch and proprioceptive feedback in visually-challenging environments. However, robots might not always know when their sensors are corrupted, as even broken sensors can return valid values. In this work, we introduce the Crossmodal Compensation Model (CCM), which can detect corrupted sensor modalities and compensate for them. CMM is a representation model learned with self-supervision that leverages unimodal reconstruction loss for corruption detection. CCM then discards the corrupted modality and compensates for it with information from the remaining sensors. We show that CCM learns rich state representations that can be used for manipulation policies learned with reinforcement learning, even when input modalities are corrupted during policy rollout in ways not seen during training.