Technical Note - Dynamic Programming and Probabilistic Constraints
Technical Note - Dynamic Programming and Probabilistic Constraints
复制标题
技术说明 - 动态规划和概率约束
作者：
D. White
本文讨论了如何使用拉格朗日乘子和有效解决方案的思想来解决涉及概率约束的动态问题。无限和有限时间范围都被考虑。在非常一般的条件下，拉格朗日乘子和有效解决方案方法将很容易通过动态规划公式产生最佳解决方案类别。然而，由此产生的约束水平可能存在间隙。结果表明，如果我们承认混合策略，这些差距就可以被填补，此外，在某些一般情况下，动态规划计算最初可以根据纯策略进行，并可以从中生成最优混合策略。概率约束以两种方式处理，即，通过考虑对系统进入特定状态的概率施加约束的情况，以及通过考虑需要最小性能方差受到平均性能约束的情况。最后从有效解理论的角度来看待均值/方差问题。可以看出，一些主要的方差最小化定理可能与这个更一般的理论有关，并且使用动态规划方法也可以获得有效的解决方案。
This note deals with the manner in which dynamic problems, involving probabilistic constraints, may be tackled using the ideas of Lagrange multipliers and efficient solutions. Both the infinite and finite time horizon are considered. Under very general conditions, Lagrange-multiplier and efficient-solution methods will readily produce, via the dynamic-programming formulations, classes of optimal solutions. However there may be gaps in the constraint levels thus generated. It is shown that, providing we admit mixed policies, these gaps can be filled in and that, furthermore, the dynamic programming calculations may, in some general circumstances, be carried out initially in terms of pure policies, and optimal mixed policies can be generated from these. The probabilistic constraints are treated in two ways, viz., by considering situations in which constraints are placed on the probabilities with which systems enter into specific states, and by considering situations in which minimum variances of performance are required subject to constraints on mean performance. Finally the mean/variance problem is viewed from the point of view of efficient solution theory. It is seen that some of the main variance-minimization theorems may be related to this more general theory, and that efficient solutions may also be obtained using dynamic-programming methods.