A Study on Discounted Markov Decision Process with Variance Criterion and its Application

在这个研究项目中，我们处理了有限状态空间和有限行动空间的马尔可夫过程和半决策过程。为了评估每一个固定的政策，我们认为期望值和方差作为标准的风险总贴现成本。在这种情况下，我们提出了最优决策问题。（1）对于带费用的连续时间马氏过程，首先给出了求方差的一组方程。在混合政策下，我们试图给出这样的方程。然而，这太难成功了。其次，我们讨论了一个连续时间马尔可夫退化系统，导出了在控制限替换策略下总折扣费用的期望值和方差。此外，数值讨论了最优控制极限状态的判定问题。（2）对于带费用的半马氏过程和更新过程，给出了求期望值和方差的方法。对于块替换策略，我们证明了方差可以通过只考虑一个周期来获得。(3)We处理了具有良好状态和磨损状态的可靠性系统，讨论了同时考虑总折扣检验费用和更换费用的期望值和方差的最优检验、老化更换问题。对最优检测时间间隔进行了数值研究。

In this research project, we treated Markov and Semi-Decision Process with a finite state space and a finite action space. To evaluate each stationary policy, we considered an expected value and variance as a criterion for risk of total discounted cost. Under this situation, we proposed optimal decision problems.(1)For a continuous time Markov process with costs, we first gave a set of equations to derive variance. Under mixed policy, we tried to give such equations. It is, however, too difficult to success. Next, we treated a continuous time Markovian deterioration system and derived expected value and variance of total discounted cost when a control limit replacement policy. Moreover, a decision problem of an optimal control limit state was numerically discussed.(2)For a semi-Markov process and renewal process with costs, we gave a method to derive expected value and variance. For a block replacement policy, we showed that variance can be obtained by considering only one cycle.(3)We treated a reliability system with a good state and a wearout state and discussed an optimal inspection, age replacement problem which takes into consideration both expected value and variance of total discounted inspection, replacement costs. An optimal inspection time interval was numerically investigated.