Beyond "To Act or Not to Act": Fast Lagrangian Approaches to General Multi-Action Restless Bandits
Beyond "To Act or Not to Act": Fast Lagrangian Approaches to General Multi-Action Restless Bandits
复制标题
超越“行动或不行动”：快速拉格朗日方法处理一般多动作不安强盗贼
DOI：
10.5555/3463952.3464038
复制
发表时间：
2021
影响因子：
--
通讯作者：
M. Tambe
中科院分区：
文献类型：
--
作者：
J. Killian;A. Perrault;M. Tambe
本文提出了新的算法和理论结果的解决方案，多行动多臂不安分的强盗，一个重要的，但研究不够的推广传统的多臂不安分的强盗（MARB）。虽然MARB在建模许多问题时很受欢迎，但它们仅限于二进制操作，即，“行动还是不行动”。这使得他们无法捕捉规划者在真实的领域中面临的关键复杂性，例如平衡维护、维修和作业调度的系统经理，或者决定给定患者治疗方法的卫生工作者。有限的以前的工作多行动MARBs只专门的子问题。在这里，我们推导出多种算法用于一般的多行动MARB使用拉格朗日松弛技术，导致以下贡献：（i）我们开发的BLam，一个边界优化算法，利用问题的凸性，快速和可证明收敛到性能良好的拉格朗日政策;（ii）我们发展了一种快速抽样技术SampleLam，用于估计拉格朗日策略，并推导出一个浓度界，以研究其收敛性质;（iii）我们得出我们的算法以及我们的主要竞争对手的最佳和最坏情况下的计算复杂性;（iv）我们提供了实验结果，将我们的算法与模拟分布的基线进行比较，包括一个由真实世界的社区健康干预任务激励的结果。我们的方法实现了显着的，高达10倍的加速比更一般的方法，而不牺牲性能，并广泛适用于一般的多动作MARB。代码可在https://github.com/killian-34/MAMARB-Lagrange-Policies上获得。
This paper presents new algorithms and theoretical results for solutions to Multi-action Multi-armed Restless Bandits, an important but insufficiently studied generalization of traditional Multi-armed Restless Bandits (MARBs). Though MARBs are popular for modeling many problems, they are restricted to binary actions, i.e., "to act or not to act". This renders them unable to capture critical complexities faced by planners in real domains, such as a system manager balancing maintenance, repair, and job scheduling, or a health worker deciding among treatments for a given patient. Limited previous work on Multi-action MARBs has only been specialized to sub-problems. Here we derive multiple algorithms for use on general Multi-action MARBs using Lagrangian relaxation techniques, leading to the following contributions: (i) We develop BLam, a bound optimization algorithm which leverages problem convexity to quickly and provably converge to the well-performing Lagrange policy; (ii) We develop SampleLam, a fast sampling technique for estimating the Lagrange policy, and derive a concentration bound to investigate its convergence properties; (iii) We derive best and worst case computational complexities for our algorithms as well as our main competitor; (iv) We provide experimental results comparing our algorithms to baselines on simulated distributions, including one motivated by a real-world community health intervention task. Our approach achieves significant, up to ten-fold speedups over more general methods without sacrificing performance and is widely applicable across general Multi-action MARBs. Code is available at https://github.com/killian-34/MAMARB-Lagrange-Policies.
排队控制和资产管理的可索引性的一般概念
DOI：
10.1214/10-aap705
发表时间：
2011
期刊：
The Annals of Applied Probability
影响因子：
--
作者：
Glazebrook K
通讯作者：
Glazebrook K