Optimal Economic Change Detection with Imperfect Information

点击翻译按钮获取中文摘要

We propose to address one of the most commonly encountered stochastic optimization problems, known as the optimal economic change detection. Briefly stated, this problem aims to detect changes in the given stochastic (reward or cost) process to maximize (or minimize) the given optimality criterion (total reward or cost). Uncertainties may come from a variety of sources including system state and its evolution, sojourn times before state change, white noise in the observable signal, and so on. The optimal change detection problem has a wide range of applications, including: production system control (Chryssolouris 2006, Zhang 2005), inventory management, healthcare system (Woodall 2006), maintenance optimization, and finance, to name just a few. Specific examples may include the dynamic production lot sizing, in which a production run will have to be stopped to minimize the production cost; the optimal inventory control, in which inventory threshold for ordering will have to be found; and optimal trading of financial assets, in which an asset position can be cleared once the price rises above a pre-set threshold. Despite the ample applications, the relevant literature is surprisingly limited. Most of the work is computational and analytical results are available only for simple cases. In particular, structural properties of optimal policy are available for problems with no more than two states. The existing literature can be extended by generalizing the basic problem (two states with complete information) in many directions: multiple system states, limited availability of information, finite or infinite optimization horizon, alternative stopping actions, and adaptive sampling. Our long-term objectives are (1) to develop computationally efficient algorithms for the whole spectrum of the economic change detection problem with imperfect information by characterizing the optimal policy and by utilizing the acceleration operators, and (2) to apply the developed model(s) to diverse applications such as maintenance optimization, production system control, financial optimization, security system control, and healthcare delivery optimization. Specific problems in the economic detection problem spectrum include (1) finite vs. infinite horizon problem, (2) N-state transition structure with or without an absorbing state(s), (3) complete (or as complete as possible) graph extension of the transition structure, (4) multiple stopping actions, (5) multiple types of samples, and (6) variable sampling interval. As a first step, our short-term objectives are to address two untapped problems in spectrum of Bayesian economic stopping problem and apply the model to an application. First, we will characterize the structure of optimal policy for the economic detection with optimal adaptive sampling and that with multiple post-change actions. These tasks will involve analytical investigation of the value function, derivation of bounds for the value function, investigation of the convexity and/or concavity of policy maps, understanding the relations among policy parameters such as super- and/or sub-modularity, and the analysis of impact of sampling rate on the value function. The eventual goal is to design computationally efficient algorithms for general classes of the economic detection problems. Second, on the application side, we will utilize the gained knowledge from the first short-term goal in tackling a real world problem: algorithmic trading. Specifically, we will extend the current development on pair trading to a more general dynamic portfolio management model. The trading problem requires continuous control as asset prices changing continuously and as a result will be tackled as continuous time Markov decision processes.