Engineering Initiation Award: Time-Average Markov Decision Processes: Theory Algorithms and Applications

本研究将研究马尔可夫决策过程（mdp），特别是与电信相关的决策过程。mdp已被提出用于各种领域，在这些领域中，决策必须在不确定性的存在下依次做出：通信网络（路由，流量控制，多路复用）；运营管理（库存控制、机器更换、高速公路养护、住院调度）；还有统计学（序贯假设检验）等等。然而，mdp的工业实现却出奇地少，特别是在电信行业。首席研究员将研究一种时间平均MDPs的新配方，以克服缺乏应用的一些明显原因。它应该导致对经典结果的更简单和更优雅的证明，并且也应该是获得具有广泛适用性的新结果和算法的强大技术。在电信系统方面，本研究将基于研究人员先前对综合通信网络的最优流量控制和最优多路复用的研究，探讨时间平均mdp在各种排队模型中的应用。还预期新的提法将大大扩大方案对电信的适用范围。研究的另一个重要方面是同时研究MDPs的具体应用以及MDPs的数学理论，从而协同提高结果的质量和数量。

Engineering Initiation Abstract - Ross This research will investigate Markov Decision Processes (MDPs), particularly as they relate to telecommunications. MDPs have been proposed for a variety of fields in which decisions must be made sequentially in the presence of uncertainty: communication networks (routing, flow control, multiplexing); operations management (inventory control, machine replacement, highway maintenance, hospital admission scheduling); and statistics (sequential hypothesis testing), among others. However, industrial implementations of MDPs are surprisingly few, particularly in the telecommunication industry. The principal investigator will study a new formulation for time-average MDPs that could overcome some of the apparent reasons for the lack of their application. It should lead to simpler and more elegant proofs for the classical results, and should also be a powerful technique for obtaining new results and algorithms with a wide range of applicability. Regarding telecommunication systems, the study will address the application of time-average MDPs to a variety of queueing models, based on the researcher's previous work with optimal flow control and optimal multiplexing for integrated communication networks. It is also anticipated that the new formulation will broaden substantially the range of applicabability of MDPs to telecommunications. Another important aspect of the research is the simultaneous study of concrete applications of MDPs along with the mathematical theory of MDPs, thereby increasing synergistically the quality and quantity of the results.