MIMO Transmission Control in Fading Channels—A Constrained Markov Decision Process Formulation With Monotone Randomized Policies
MIMO Transmission Control in Fading Channels—A Constrained Markov Decision Process Formulation With Monotone Randomized Policies
复制标题
衰落信道中的 MIMO 传输控制——具有单调随机策略的约束马尔可夫决策过程公式
DOI：
10.1109/tsp.2007.897859
复制
发表时间：
2007
影响因子：
5.4
通讯作者：
V. Krishnamurthy
中科院分区：
文献类型：
--
作者：
D. Djonin;V. Krishnamurthy
本文讨论了马尔可夫衰落信道上多输入多输出 (MIMO) 无线系统中的最优功率和速率分配控制。该问题被提出为无限水平平均成本约束马尔可夫决策过程（CMDP），其目标是最小化受延迟约束的平均传输功率。通过使用 CMDP 的拉格朗日公式，我们使用随机优势、子模性和多模性的概念来证明最优随机策略是单调的。得出了关于最优随机政策性质的三个重要的结构结果。首先，我们表明，通过将速率分配问题分解为跨单个天线的比特加载问题以及基于当前缓冲区占用和信道状态的总速率分配，可以以指数方式减少操作空间。其次，我们证明最优速率分配策略是缓冲区占用率单调增加的两种纯策略的随机混合。最后，我们证明最优功率分配在延迟约束中是分段线性的。可以利用这三个结构结果来设计有效的在线强化学习算法以实现最佳速率分配。
This paper addresses the optimal power and rate allocation control in multiple-input multiple-output (MIMO) wireless systems over Markovian fading channels. The problem is posed as an infinite horizon average-cost constrained Markov decision process (CMDP) with the goal of minimizing the average transmission power subject to delay constraints. By using a Lagrangian formulation of the CMDP, we use the concepts of stochastic dominance, submodularity, and multimodularity to prove that the optimal randomized policies are monotone. Three important structural results on the nature of the optimal randomized policies are derived. First, we show that the action space can be exponentially reduced by decomposing the rate allocation problem into bit-loading problem across individual antennas and the total rate allocation based on the current buffer occupancy and channel state. Second, we show that the optimal rate allocation policy is a randomized mixture of two pure policies that are monotonically increasing in the buffer occupancy. Finally, we show that the optimal power allocation is piecewise linear in the delay constraint. These three structural results can be exploited to devise efficient online reinforcement learning algorithms for optimal rate allocation.