Towards Applying Interactive POMDPs to Real-World Adversary Modeling
Towards Applying Interactive POMDPs to Real-World Adversary Modeling
复制标题
DOI：
10.1609/aaai.v24i2.18818
复制
发表时间：
2010-07
期刊：
影响因子：
--
通讯作者：
Brenda Ng;C. Meyers;K. Boakye;J. Nitao
中科院分区：
文献类型：
--
作者：
Brenda Ng;C. Meyers;K. Boakye;J. Nitao
我们研究了使用决策过程来模拟现实世界中的智能对手系统的适用性。长期以来，决策过程一直被用来研究合作多智能体相互作用，但其实际适用性的对抗性问题已收到最少的研究。我们解决了在这一领域应用顺序决策的利弊，使用洗钱犯罪作为一个具体的例子。受案例研究的启发，我们抽象出一个模型的洗钱过程中，使用的框架交互式部分可观察马尔可夫决策过程（I-POMDPs）。我们解决了为什么这个框架非常适合建模对抗性交互。粒子滤波和值迭代被用来解决模型，与不同的修剪和前瞻策略的应用程序，以评估解决方案的质量和算法的运行时间之间的权衡。我们的研究结果表明，有一个很大的差距，目前可以实现这样的决策模型，主要是由于计算需求，限制了可以解决的问题的大小的现实主义水平。虽然这些结果代表了一个简化的洗钱模型的解决方案，但它们说明了代理人之间的相互作用，不能被捕获的标准方法，如异常检测。这意味着，I-POMDP方法可能是有价值的，在未来，当算法能力进一步发展。
We examine the suitability of using decision processes to model real-world systems of intelligent adversaries. Decision processes have long been used to study cooperative multiagent interactions, but their practical applicability to adversarial problems has received minimal study. We address the pros and cons of applying sequential decision-making in this area, using the crime of money laundering as a specific example. Motivated by case studies, we abstract out a model of the money laundering process, using the framework of interactive partially observable Markov decision processes (I-POMDPs). We address why this framework is well suited for modeling adversarial interactions. Particle filtering and value iteration are used to solve the model, with the application of different pruning and look-ahead strategies to assess the tradeoffs between solution quality and algorithmic run time. Our results show that there is a large gap in the level of realism that can currently be achieved by such decision models, largely due to computational demands that limit the size of problems that can be solved. While these results represent solutions to a simplified model of money laundering, they illustrate nonetheless the kinds of agent interactions that cannot be captured by standard approaches such as anomaly detection. This implies that I-POMDP methods may be valuable in the future, when algorithmic capabilities have further evolved.