Engineering Initiation Award: Time-Average Markov Decision Processes: Theory Algorithms and Applications

点击翻译按钮获取中文摘要

Engineering Initiation Abstract - Ross This research will investigate Markov Decision Processes (MDPs), particularly as they relate to telecommunications. MDPs have been proposed for a variety of fields in which decisions must be made sequentially in the presence of uncertainty: communication networks (routing, flow control, multiplexing); operations management (inventory control, machine replacement, highway maintenance, hospital admission scheduling); and statistics (sequential hypothesis testing), among others. However, industrial implementations of MDPs are surprisingly few, particularly in the telecommunication industry. The principal investigator will study a new formulation for time-average MDPs that could overcome some of the apparent reasons for the lack of their application. It should lead to simpler and more elegant proofs for the classical results, and should also be a powerful technique for obtaining new results and algorithms with a wide range of applicability. Regarding telecommunication systems, the study will address the application of time-average MDPs to a variety of queueing models, based on the researcher's previous work with optimal flow control and optimal multiplexing for integrated communication networks. It is also anticipated that the new formulation will broaden substantially the range of applicabability of MDPs to telecommunications. Another important aspect of the research is the simultaneous study of concrete applications of MDPs along with the mathematical theory of MDPs, thereby increasing synergistically the quality and quantity of the results.