Studies on optimal theory and its application in probabilistic decision processes with general utility functions.

停止决策过程是马尔可夫决策过程和停止问题的组合模型。MDP由可数状态集S、紧动作空间A（i）、转移概率q=（qa<ij>）和一致有界的立即报酬函数r（i，a，j）定义，它们对任意i，j ∈ S在a ∈ A（i）中连续.策略π是A（i_t）上的概率序列，由每个历史（i_0，a_0，i_1，.，i_t）（t= 0，1，.）约束.用σ表示停时，g表示效用函数，设B（t）= n ^t_&lt;k=1&gt; r（X_<k-1>，Δ_<k-1>，X_k），其中X_t和Δ_t分别是t时刻的状态和动作。（π，σ）对称为（i_0，α_0）-最优的，如果它使E^π_<i_0>[g（α_0+B（σ））]最大化，其中E^π_<i_0>[g（α_0+B（t））]是初始状态i_0在样本空间Ω=（S×A）^∞上的概率测度的期望。假设g是非减的、凹的且上有界的，或者g在真实的线R的任何紧子集上有界导数，对任意π，i，其中g^+是g的正部分。设v（i，α）= max_&lt;{（π，σ）}&gt; E^π_i（g（α+B（σ））.然后，我们得到了以下结果。1.对任意i ∈ S和α，α（i，α）满足最优性方程&lt;$（i，α）= max {g（α），max_&lt;α∈A&gt;&lt;$_&lt;j∈S&gt; q_<ij>（a）&lt;$（j，α+r（i，a，j）}（1）进一步，设（π，σ）满足P^π_<i_0>（σ&gt;1）=1.2.若（π，σ）是（i_0，α_0）-最优对，则E^π_<i_0>[g（α_0 + B（σ））]满足（1）.若E^π_<i_0>[g（α_0 + B（σ））]满足（1），则（π，σ）是（i_0，α_0）-最优的.

Stopped decision process is a combined model of Markov decision processes (MDPs) and the stopping problem. MDPs are specified by the set of countable states S, compact action space A (i) assigned at each state i ∈ S, transition probabilities q= (q_<ij>(a)), and a uniformly bounded immediate reward function r(i, a, j), which are continuous in a ∈ A(i) for any i, j ∈ S.A policy π is a sequence of probabilities on A (i_t) conditioned by each histories (i_0, a_0, i_1, …, i_t ) for t=0,1, …. Denote by σ a stopping time and by g a utility function.Let B(t)=Σ^t_<k=1> r (X_<k-1>, Δ_<k-1>, X_k), where X_t and Δ_t are the state and action at time t, respectively. The pair (π, σ) is called (i_0, α_0)-optimal if it maximizes E^π_<i_0> [g (α_0+B(σ))], where E^π_<i_0> is the expectation by the probability measure on the sample space Ω=(S×A)^∞ for an initial state i_0.It is assumed that g is non-decreasing, concave and bounded above, or that g has an bounded derivative on any compact subset of the real line R satisfying E^π_i [sup_<t【greater than or equal】0> g^+(α_0+B(t))] < ∞ for any π, i, where g^+ is the positive part of g. Let v(i, α) = max_<{(π, σ)}> E^π_i (g(α+B(σ)). Then, we have following results.1. For any i ∈ S and α, υ(i, α) satisfies optimality equationsυ(i, α) = max {g(α), max_<α∈A> Σ_<j∈S> q_<ij>(a) υ (j, α+r(i, a, j)}(1)Furthermore, suppose (π, σ) satisfies P^π_<i_0> (σ>1)=1.2. If (π, σ) is (i_0, α_0)-optimal pair, then E^π_<i_0> [g(α_0 + B(σ))] satisfies (1).3. If E^π_<i_0>[g(α_0 + B(σ))] satisfies (1), then (π, σ) is (i_0, α_0)-optimal.