Comparing Action-Query Strategies in Semi-Autonomous Agents
Comparing Action-Query Strategies in Semi-Autonomous Agents
复制标题
比较半自主代理中的操作查询策略
DOI：
10.1609/aaai.v25i1.7992
复制
发表时间：
2011
期刊：
影响因子：
--
通讯作者：
Satinder Singh
中科院分区：
文献类型：
--
作者：
Robert S. Cohn;E. Durfee;Satinder Singh
我们考虑的设置是，一个半自主的代理对其环境有不确定的知识，但可以询问人类操作员在当前或潜在的未来状态下更愿意采取什么行动。请求查询可以改善行为，但是如果查询是有代价的（例如，由于操作人员的注意力有限），则应该最大化每个查询的价值。我们比较了选择动作查询的两种策略：1)基于短视最大化长期价值的期望收益，2)基于短视最小化代理策略表示中的不确定性。我们的经验表明，第一种策略倾向于选择更有价值的查询，并且混合方法可以在计算有限的情况下优于单独使用任何一种方法。
We consider settings in which a semi-autonomous agent has uncertain knowledge about its environment, but can ask what action the human operator would prefer taking in the current or in a potential future state. Asking queries can improve behavior, but if queries come at a cost (e.g., due to limited operator attention), the value of each query should be maximized. We compare two strategies for selecting action queries: 1) based on myopically maximizing expected gain in long-term value, and 2) based on myopically minimizing uncertainty in the agent's policy representation. We show empirically that the first strategy tends to select more valuable queries, and that a hybrid method can outperform either method alone in settings with limited computation.