Reinforcement Learning for Automated Flowsheet Synthesis of Steady-State Processes

点击翻译按钮获取中文摘要

Flowsheet synthesis is a key step in conceptual design of chemical processes. By its nature, it is a creative process that is hard to formalize. Current methods of computer-aided flowsheet synthesis are however mostly formalized algorithms that employ knowledge-based rules for creating flowsheet alternatives and mathematical programming for selecting optimal flowsheets from larger sets of alternatives (typically defined via superstructures). The goal of this project is using the recently achieved progress in reinforcement learning (RL) for automated but creative (here: inventive and explorative) flowsheet synthesis of steady-state chemical processes. The RL environment is a process simulator that contains the a priori physical knowledge, i.e. physico-chemical property data and a set of general process unit models. Step by step, the RL agent can set up process flowsheets, modify them, and evaluate them in the process simulator to obtain feedback/reward. The agent has no prior knowledge of chemical engineering and is trained to create flowsheets solely through automated interaction with the simulator. The central work hypothesis is that this setup is able to autonomously produce feasible process flowsheets that are near-optimal or optimal regarding a given cost function.In the first period of the Priority Programme, the Burger group (Chemical Engineering) will develop and implement simulation environments for several example problems. Simplified shortcut and surrogate models for process units are used to obtain robust simulation environments. Despite the reduced model depth, the action space remains large (due to a large number of conceivable flowsheet variants) and parameterized (due to continuous parameters of the process units). The Grimm Group (Machine Learning) will develop tailored RL methods to cope with these challenges. Hierarchical RL and novel methods for parametrized action spaces will be investigated and developed. For improved forward planning and exploration, the flowsheet synthesis problem will be embedded into a competitive two-player game setup, which has been introduced in joint preliminary work. This allows for efficient training in self-play using algorithms developed for classical game applications (Chess, Go). Close collaboration between both groups is required in all project stages to find optimal rules of the two-player game (allowed actions, objectives and reward) and optimal agent structures (feature selection, hierarchical decisions).The project is located in field F of the Priority Programme’s collaboration matrix and primarily in the research area #6 creativity. Great collaboration potential with other projects is given, because this project shares common interests with all projects that will develop robust simulation environments, creatively design process units or molecular structures, and/or optimize processes.