Autonomous Control of Urban Storm Water Networks Using Reinforcement Learning
Autonomous Control of Urban Storm Water Networks Using Reinforcement Learning
复制标题
使用强化学习的城市雨水网络自主控制
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
B. Kerkez
中科院分区：
文献类型：
--
作者：
Abhiram Mullapudi;B. Kerkez
我们研究了 12 公里长的城市雨水管网的实时自主运行情况，该管网已安装了传感器和控制阀。具体来说，我们将强化学习（一种植根于深度学习的技术）评估为系统级控制方法。控制器打开和关闭系统中的阀门，通过协调空间分布的雨水资产（即滞留盆地和湿地）之间的排放来增强雨水网络的性能。采用强化学习控制算法来控制城市流域的雨水网络。结果表明，使用强化学习的阀门控制显示出巨大的潜力，但仍需要进行广泛的研究以形成对控制鲁棒性的基本理解。我们特别讨论了奖励函数（即启发式控制目标）的作用和重要性，它指导自主控制器实现所需的流域规模响应。
We investigate the real-time and autonomous operation of a 12 km urban storm water network, which has been retrofitted with sensors and control valves. Specifically, we evaluate reinforcement learning, a technique rooted in deep learning, as a system-level control methodology. The controller opens and closes valves in the system, which enhances the performance in the storm water network by coordinating the discharges amongst spatially distributed storm water assets (i.e. detention basins and wetlands). A reinforcement learning control algorithm is implemented to control the storm water network across an urban watershed. Results show that control of valves using reinforcement learning shows great potential, but extensive research still needs to be conducted to develop a fundamental understanding of control robustness. We specifically discuss the role and importance of the reward function (i.e. heuristic control objective), which guides the autonomous controller towards achieving the desired water shed scale response.