Simulation and Function Approximation Based Iterative Approach To Process Control

研究：PI计划研究一种基于模拟和函数近似的策略，为控制策略带来进化改进。由于当前基于模型的预测控制公式在处理大规模复杂动力学和大量不确定性的系统方面存在一些固有的局限性，因此进行了这项研究。这一发展将植根于人工智能领域的一种方法，这种方法被称为神经动态规划和强化学习，在处理高度复杂的多阶段离散决策问题（如双陆棋、电梯调度问题和作业车间调度问题）方面取得了巨大成功。当外推到过程控制问题时，该方法首先对一组广泛的可能操作条件使用给定的次优控制策略进行闭环模拟。然后，模拟结果用于生成状态与“成本-走”或“奖励”函数的数据，通常是通过将神经网络拟合到数据中。通过额外的离线计算，通过基于迭代Bellman方程的“值迭代”或基于在策略评估和策略改进之间迭代的“策略迭代”，可以改进近似。通过将大视界问题简化为等效的短视界问题，或者允许对改进的控制律进行离线参数化，改进后的“走成本”函数的近似值可用于以计算效率高的方式实现最优控制。为了使该方法在过程控制中切实可行，需要解决一些问题。该方法的成功将取决于是否能够获得成本函数的精确和鲁棒近似。一个直接要问的问题是什么类型的函数逼近器最适合于近似。此外，在控制计算中需要考虑通过内插和外推的神经网络成本预测的置信度。PI计划调查这些和其他基本问题，以得出系统和实际有用的答案。PI将与工业伙伴Weyerhauser、Owens Corning、LG化学和Aspen Technology合作，在实际工业过程中测试开发的工具，并将其纳入商业过程控制软件包。项目的申请部分将由这些公司支持的学生和访客进行。更广泛的影响：化学过程工业（CPI）充满了涉及重大不确定性的非线性控制问题，这可以从这项工作中受益。除了过程控制外，该策略还适用于不确定情况下的计划和调度问题以及供应链运行问题。

Research:The PI plans to investigate a simulation and function approximation-based strategy for bringing an evolutionary improvement to a control policy. The investigation is motivated by some inherent limitations of the current model-based predictive control formulation with respect to handling systems of large-scale complex dynamics, and large amount of uncertainty. The development will be rooted in an approach developed in the field of artificial intelligence - referred to by various names such as Neuro-Dynamic Programming and Reinforced Learning - which has shown great success in handling highly complex multi-stage discrete decision problems like backgammon playing, elevator dispatch problem, and job-shop scheduling. The approach, when extrapolated to the problem of process control, begins by performing closed-loop simulations with a given suboptimal control policy for an extensive set of possible operating conditions. The simulation results are then used to generate data for state versus "cost-to-go" or "reward" function, typically by fitting a neural network to the data. The approximation is improved by additional off-line calculations, either by "value iteration" based on iterating the Bellman Equation or by "policy iteration" based on iterating between policy evaluation and policy improvement. The improved approximation of the "cost-to-go" function can be used to implement optimal control in a computationally efficient way, either by reducing a large-horizon problem into an equivalent short-horizon problem or by allowing an off-line parameterization of the improved control law. To make the approach practicable for process control a number of issues need to be resolved. The success of the approach will depend on the ability to obtain an accurate and robust approximation of the cost-to-go function. An immediate question to ask is what types of function approximators are best suited for the approximation. Also, the level of confidence in the neural network's cost predictions through interpolation and extrapolation need to be taken into account in the control calculations. The PI plans to investigate these and other fundamental issues to arrive at systematic and practically useful answers. The PI will collaborate with industrial partners Weyerhauser, Owens Corning, LG Chemicals, and Aspen Technology, to test the developed tools on real industrial process and to incorporate them into commercial process control software packages. The application portion of the project will be carried out by students and visitors supported by these companies.Broader Impact:The Chemical Process Industries (CPI) are replete with nonlinear control problems involving significant uncertainties, which can benefit from this work. In addition to process control, the strategy fits naturally to planning and scheduling problems under uncertainty as well as supply chain operation problems.