Sequential Resource Allocation in Linear Stochastic Bandits
Sequential Resource Allocation in Linear Stochastic Bandits
复制标题
线性随机强盗中的顺序资源分配
DOI：
--
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
Marta Soare
中科院分区：
文献类型：
--
作者：
Marta Soare
本论文致力于研究不确定环境中的资源分配问题，其中智能体可以顺序地选择要采取的行动。在每一步之后，环境返回所选动作的值的噪声观测。这些观察指导智能体调整其资源分配策略，以达到给定的目标。在这种最典型的设置中，随机多臂强盗（MAB），假设每个观察是从与所选动作相关联的未知概率分布中得出的，并且没有给出关于其他动作的期望值的信息。MAB设置已被广泛研究，并提出了最优分配策略，以解决各种目标下的MAB假设。在这里，我们考虑MAB设置的一个变体，其中环境中存在全局线性结构，并且通过选择一个动作，代理还收集关于其他动作的值的信息。因此，智能体需要调整其资源分配策略以利用环境中的结构。特别是，我们研究了代理应该采取的行动序列的设计，以达到目标，如：（i）确定最佳值与固定的信心，并使用最少数量的拉，或（ii）最大限度地减少每个动作的值的预测误差。此外，我们研究如何在一个给定的环境中的强盗算法收集的知识可以转移，以提高在其他类似的环境中的性能。
This thesis is dedicated to the study of resource allocation problems in uncertain environments, where an agent can sequentially select which action to take. After each step, the environment returns a noisy observation of the value of the selected action. These observations guide the agent in adapting his resource allocation strategy towards reaching a given objective. In the most typical setting of this kind, the stochastic multi-armed bandit (MAB), it is assumed that each observation is drawn from an unknown probability distribution associated with the selected action and gives no information on the expected value of the other actions. The MAB setting has been widely studied and optimal allocation strategies were proposed to solve various objectives under the MAB assumptions. Here, we consider a variant of the MAB setting where there exists a global linear structure in the environment and by selecting an action, the agent also gathers information on the value of the other actions. Therefore, the agent needs to adapt his resource allocation strategy to exploit the structure in the environment. In particular, we study the design of sequences of actions that the agent should take to reach objectives such as: (i) identifying the best value with a fixed confidence and using a minimum number of pulls, or (ii) minimizing the prediction error on the value of each action. In addition, we investigate how the knowledge gathered by a bandit algorithm in a given environment can be transferred to improve the performance in other similar environments.