Reinforcement Learning for Automated Flowsheet Synthesis of Steady-State Processes

稳态过程自动流程图合成的强化学习

• 批准号: 466387255
• 负责人: Professor Dr.-Ing. Jakob Burger
• 金额: --
• 依托单位: Fachgebiet für Bioinformatik
• 依托单位国家: 德国
• 项目类别: Priority Programmes
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/466387255?language=en
• 关键词: Reinforcement; Learning; Automated; Flowsheet; Synthesis

Flowsheet synthesis is a key step in conceptual design of chemical processes. By its nature, it is a creative process that is hard to formalize. Current methods of computer-aided flowsheet synthesis are however mostly formalized algorithms that employ knowledge-based rules for creating flowsheet alternatives and mathematical programming for selecting optimal flowsheets from larger sets of alternatives (typically defined via superstructures). The goal of this project is using the recently achieved progress in reinforcement learning (RL) for automated but creative (here: inventive and explorative) flowsheet synthesis of steady-state chemical processes. The RL environment is a process simulator that contains the a priori physical knowledge, i.e. physico-chemical property data and a set of general process unit models. Step by step, the RL agent can set up process flowsheets, modify them, and evaluate them in the process simulator to obtain feedback/reward. The agent has no prior knowledge of chemical engineering and is trained to create flowsheets solely through automated interaction with the simulator. The central work hypothesis is that this setup is able to autonomously produce feasible process flowsheets that are near-optimal or optimal regarding a given cost function.In the first period of the Priority Programme, the Burger group (Chemical Engineering) will develop and implement simulation environments for several example problems. Simplified shortcut and surrogate models for process units are used to obtain robust simulation environments. Despite the reduced model depth, the action space remains large (due to a large number of conceivable flowsheet variants) and parameterized (due to continuous parameters of the process units). The Grimm Group (Machine Learning) will develop tailored RL methods to cope with these challenges. Hierarchical RL and novel methods for parametrized action spaces will be investigated and developed. For improved forward planning and exploration, the flowsheet synthesis problem will be embedded into a competitive two-player game setup, which has been introduced in joint preliminary work. This allows for efficient training in self-play using algorithms developed for classical game applications (Chess, Go). Close collaboration between both groups is required in all project stages to find optimal rules of the two-player game (allowed actions, objectives and reward) and optimal agent structures (feature selection, hierarchical decisions).The project is located in field F of the Priority Programme’s collaboration matrix and primarily in the research area #6 creativity. Great collaboration potential with other projects is given, because this project shares common interests with all projects that will develop robust simulation environments, creatively design process units or molecular structures, and/or optimize processes.

流程图综合是化工过程概念设计的关键步骤。就其性质而言，这是一个很难正式化的创造性过程。然而，计算机辅助流程图合成的当前方法大多是形式化的算法，其采用基于知识的规则来创建流程图备选方案和数学编程来从更大的备选方案集合（通常经由超结构定义）中选择最优流程图。该项目的目标是利用最近在强化学习（RL）方面取得的进展，实现稳态化学过程的自动化但具有创造性（此处：创造性和探索性）的流程图合成。RL环境是一个过程模拟器，它包含先验物理知识，即物理化学性质数据和一组通用过程单元模型。一步一步地，RL代理可以建立过程流程图，修改它们，并在过程模拟器中评估它们以获得反馈/奖励。代理人没有化学工程的先验知识，并接受培训，仅通过与模拟器的自动交互来创建流程图。核心工作假设是，这种设置能够自主地产生可行的工艺流程，这些流程在给定的成本函数方面接近最优或最优。在优先计划的第一阶段，Burger集团（化学工程）将为几个示例问题开发和实施模拟环境。简化的捷径和代理模型的过程单元被用来获得强大的仿真环境。尽管模型深度降低，但动作空间仍然很大（由于大量可想到的流程图变体）和参数化（由于过程单元的连续参数）。Grimm Group（机器学习）将开发量身定制的RL方法来应对这些挑战。将研究和开发用于参数化动作空间的分层强化学习和新方法。为了改进前瞻性规划和探索，流程图综合问题将嵌入到一个竞争性的两个玩家的游戏设置，这已被引入联合前期工作。这允许使用为经典游戏应用（国际象棋、围棋）开发的算法进行自我游戏的有效训练。在项目的所有阶段都需要两个小组之间的密切合作，以找到两个玩家游戏的最佳规则（允许的行动、目标和奖励）和最佳代理结构（特征选择、分层决策）。该项目位于优先方案合作矩阵的F字段，主要是在研究领域#6创造力。与其他项目的合作潜力很大，因为该项目与所有将开发强大的模拟环境，创造性地设计工艺单元或分子结构和/或优化工艺的项目有着共同的利益。