A Study on Discounted Markov Decision Process with Variance Criterion and its Application

点击翻译按钮获取中文摘要

In this research project, we treated Markov and Semi-Decision Process with a finite state space and a finite action space. To evaluate each stationary policy, we considered an expected value and variance as a criterion for risk of total discounted cost. Under this situation, we proposed optimal decision problems.(1)For a continuous time Markov process with costs, we first gave a set of equations to derive variance. Under mixed policy, we tried to give such equations. It is, however, too difficult to success. Next, we treated a continuous time Markovian deterioration system and derived expected value and variance of total discounted cost when a control limit replacement policy. Moreover, a decision problem of an optimal control limit state was numerically discussed.(2)For a semi-Markov process and renewal process with costs, we gave a method to derive expected value and variance. For a block replacement policy, we showed that variance can be obtained by considering only one cycle.(3)We treated a reliability system with a good state and a wearout state and discussed an optimal inspection, age replacement problem which takes into consideration both expected value and variance of total discounted inspection, replacement costs. An optimal inspection time interval was numerically investigated.