Studies on Learning Algorithms for Flexibly Structured Decision Process Models

在这个项目中，我们的目标是为不确定决策过程建立具有更灵活和更软结构的自适应和强化学习算法。主要研究成果如下。1.柔性结构模型的构建与分析研究(A)研究了模糊性的可能性和可信度，并应用其延拓定理，从给定的条件可信度度量出发，成功地构造了可信度过程，为模糊环境下决策过程的公理化发展提供了可能。(B)我们成功地导出了具有一般效用函数的吸收半马尔可夫对策的柔性最优性方程，该方程决定了最优策略。对于一个质量控制问题的贝叶斯分析，我们提出了一种新的控制图，它具有更灵活的结构，通过测量的先验区间来掌握未知参数。2.自适应马尔可夫决策模型的学习算法我们提出了一种自适应马尔可夫决策模型的模式矩阵学习算法，它从观测数据中学习转移矩阵的结构(模式)，并利用其信息构造基于时间差分(TD)方法的自适应策略。这种方法本质上可以适用于多链情况。3.强化学习方法的应用(A)研究了适用于各种神经元动态规划模型的TD或Actor Critic算法的收敛性，并通过数值实验证明了其有效性。(2)为了求解模糊环境下的多种运筹学模型，提出了一种模糊模拟与遗传算法相结合的混合智能算法，并通过算例验证了该算法的有效性。

In this project, our objective is to establish the adaptive and reinforcement learning algorithms for uncertain decision processes with the more flexible and soft structure. The main research results are as follows. 1. Further studies on construction and analysis of flexibly structured models (a) Investigating possibility and credibility of fuzziness and applying its extension theorem, we have succeeded in constructing credibilistic process from given conditional credibility measures, by which axiomatic development of decision processes under fuzzy environment will be made to be possible. (b) We have succeeded in deriving the flexible optimality equations for an absorbing semi-Markov game with general utility functions which determine the optimal strategies. c Concerning Bayesian analysis for a quality control problem, we have proposed the new control chart which has more flexible structure, grasping the unknown parameter by a priori interval of measures. The efficiency of the new one is shown by comparing with the usual one 2. Learning algorithms for adaptive Markov decision models (MDPs) We have developed a pattern-matrix learning algorithm for adaptive MDPs which learns the structure (pattern) of transition matrices from the observed data and using its information constructs the adaptive policy based on temporal difference (TD) method. This method can be essentially applicable to the multichain case. 3. Application of reinforcement learning methods (a) We have investigated the convergence of the TD or Actor Critic algorithms applicable to various models of neuron dynamic programming, finding its efficiency by numerical experiments. (b) In order to solve several Operational Research models under fuzzy environments, we have developed the Hybrid Intelligent algorithm integrating fuzzy simulation and genetic algorithm, whose efficiency is verified by a numerical examples.