On study of temporal difference method in decision process and its application

在具有不确定性的决策过程中，利用动态规划（DP）算法构造了最优解。为了解决涉及非常大状态空间的实际问题，我们需要减少学习算法所需的计算量，因为DP算法不能直接应用。在利用状态-作用过程数据的基础上，通过学习算法对值函数进行估计。研究了具有未知过渡律的马尔可夫决策过程的神经动态规划（neural - dp）和贝叶斯区间估计的时间差分方法。从理论上推导了构造最优解的算法。通过数值算例说明了算法的有效性，并对相应的算法进行了改进。

In decision process with uncertainty, the optimal solutions are constructed by using dynamic programming(DP) algorithms. In order to solve practical problems involving very large state space, we need to decrease the amount of computation necessary for learning algorithm since DP algorithm cannot be applied directly. Based on using the data of state-action process, the value function is estimated by learning algorithm. We consider temporal difference method of Neuro Dynamic Programming (Neuro-DP) and Bayesian interval estimation in Markov decision processes with unknown transition law. We derive algorithms of constructing optimal solution theoretically. We also treat numerical examples to show the validity of algorithms and improve corresponding algorithms.