An influent responsive control strategy with machine learning: Q-learning based optimization method for a biological phosphorus removal system
An influent responsive control strategy with machine learning: Q-learning based optimization method for a biological phosphorus removal system
复制标题
机器学习进水响应控制策略：基于Q学习的生物除磷系统优化方法
DOI：
10.1016/j.chemosphere.2019.06.103
复制
发表时间：
2019-11-01
期刊：
影响因子：
8.8
通讯作者：
Ren, Nan-Qi
中科院分区：
文献类型：
--
作者：
Pang, Ji-Wei;Yang, Shan-Shan;Ren, Nan-Qi
生物除磷（BPR）是一种经济且可持续的废水除磷工艺，通过厌氧和好氧（An/Ae）工艺循环活性污泥来实现。然而，很少有研究系统地分析了厌氧和好氧反应中的最佳水力停留时间（HRT），或者这些是否是最合适的控制策略。在这项研究中，开发了一种新的优化方法，使用改进的Q-学习（QL）算法，优化在BPR系统中的An/Ae HRT。建立了基于QL的BPR控制策略框架，推导了改进的Q函数Q（t+1）（s（t），s（t+1））= Q（t）（s（t），s（t+1））+ k [R（s（t），s（t+1））+ gamma maxQ（t）（s（t），s（t+1））- Q（t）（s（t），s（t+1））].基于改进的Q函数和在不同HRT步长下得到的状态转移矩阵，可以得到任意BPR系统中An/Ae过程HRT的最优组合，即HRT的有序对组合。通过应用六种不同的进水化学需氧量（COD）浓度，从150到600毫克L-1和进水磷浓度，从12到30毫克L-1变化进行模型验证。采用最优控制策略，观察到上级和稳定的出水水质。这表明，所提出的新的基于QL的BPR模型进行适当的和派生的Q函数成功地实现了实时建模，稳定的最优控制策略下波动的进水负荷在污水处理过程中。(C)2019爱思唯尔有限公司版权所有。
Biological phosphorus removal (BPR) is an economical and sustainable processes for the removal of phosphorus (P) from wastewater, achieved by recirculating activated sludge through anaerobic and aerobic (An/Ae) processes. However, few studies have systematically analyzed the optimal hydraulic retention times (HRTs) in anaerobic and aerobic reactions, or whether these are the most appropriate control strategies. In this study, a novel optimization methodology using an improved Q-learning (QL) algorithm was developed, to optimize An/Ae HRTs in a BPR system. A framework for QL-based BPR control strategies was established and the improved Q function, Q(t+1) (s(t), s(t+1)) = Q(t)(s(t), s(t+1)) + k [R(s(t), s(t+1)) + gamma maxQ(t) (s(t), s(t+1)) - Q(t) (s(t), s(t+1))] was derived. Based on the improved Q function and the state transition matrices obtained under different HRT step-lengths, the optimum combinations of HRTs in An/Ae processes in any BPR system could be obtained, in terms of the ordered pair combinations of the . Model verification was performed by applying six different influent chemical oxygen demand (COD) concentrations, varying from 150 to 600 mg L-1 and influent P concentrations, varying from 12 to 30 mg L-1. Superior and stable effluent qualities were observed with the optimal control strategies. This indicates that the proposed novel QL-based BPR model performed properly and the derived Q functions successfully realized real-time modelling, with stable optimal control strategies under fluctuant influent loads during wastewater treatment processes. (C) 2019 Elsevier Ltd. All rights reserved.