Towards Uniformly Superhuman Autonomy via Subdominance Minimization
Towards Uniformly Superhuman Autonomy via Subdominance Minimization
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Brian D. Ziebart;Sanjiban Choudhury;Xinyan Yan;Paul Vernaza
中科院分区：
文献类型：
--
作者：
Brian D. Ziebart;Sanjiban Choudhury;Xinyan Yan;Paul Vernaza
流行的模仿学习方法寻求产生与人类平均表现相匹配或超过平均水平的行为。这通常阻碍了在确定更好的示范来模仿fi邪教时实现专家级或超人的表现。取而代之的是，我们假设演示的质量各不相同，并试图诱导出明显好于所有人类演示的行为(即，帕累托主导或最低限度的次优)。我们的最小劣势逆最优控制训练目标主要是通过高质量的演示来消除fiNed；更容易被控制的低质量的演示被有效地忽略，而不是有辱人格的模仿。随着概率的增加，我们的方法产生超人行为的成本比演示者未知成本函数上的演示成本更低-即使每个演示的成本函数不同。我们将我们的方法应用于计算机光标指向任务，产生的行为是78%的超人行为，而最小化演示次优可以提供50%的超人行为-即使在选择性数据清理之后也只有72%。
Prevalent imitation learning methods seek to produce behavior that matches or exceeds average human performance. This often prevents achieving expert-level or superhuman performance when identifying the better demonstrations to imitate is difficult. We instead assume demonstrations are of varying quality and seek to induce behavior that is unambiguously better (i.e., Pareto dominant or minimally subdominant) than all human demonstrations. Our minimum subdominance inverse optimal control training objective is primarily defined by high quality demonstrations; lower quality demonstrations, which are more easily dominated, are effectively ignored instead of degrading imitation. With increasing probability, our approach produces superhuman behavior incurring lower cost than demonstrations on the demonstrator’s unknown cost function—even if that cost function differs for each demonstration. We apply our approach on a computer cursor pointing task, producing behavior that is 78% su-perhuman, while minimizing demonstration sub-optimality provides 50% superhuman behavior— and only 72% even after selective data cleaning.