COVID-19 Pandemic Cyclic Lockdown Optimization Using Reinforcement Learning
COVID-19 Pandemic Cyclic Lockdown Optimization Using Reinforcement Learning
复制标题
使用强化学习的 COVID-19 大流行循环锁定优化
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Lyudmil Pelov
中科院分区：
文献类型：
--
作者：
M. Arango;Lyudmil Pelov
这项工作研究了使用强化学习(RL)来优化循环封锁，这是可用于控制新冠肺炎大流行的方法之一。该问题被构造为用于跟踪参考值的最优控制系统，该参考值对应于关键资源的最大使用水平，例如ICU床位。然而，与传统的最优控制方法不同，RL被用来寻找最优控制策略。开发了一种使用基于RL的开关控制器来计算最优循环锁定时间的框架。基于RL的控制器被实现为与流行病模拟器交互的RL代理，该流行病模拟器被实现为扩展的SEIR流行病模型。RL代理学习策略函数，该策略函数产生开放/锁定决策的最佳序列，使得在RL奖励函数中指定的目标被优化。同时使用了两个目标：第一个是公共卫生目标，将超过ICU床位门槛的ICU床位使用量降至最低；第二个是社会经济目标，最大限度地减少被封锁的时间。假设当一个区域面临着超过资源容量限制的迫在眉睫的危险，并且由于在长期封锁期间缺乏必要的经济资源来支持其受影响的人口而实施长期封锁将造成严重的社会和经济后果时，循环封锁被视为延长封锁的临时替代办法。
This work examines the use of reinforcement learning (RL) to optimize cyclic lockdowns, which is one of the methods available for control of the COVID-19 pandemic. The problem is structured as an optimal control system for tracking a reference value, corresponding to the maximum usage level of a critical resource, such as ICU beds. However, instead of using conventional optimal control methods, RL is used to find optimal control policies. A framework was developed to calculate optimal cyclic lockdown timings using an RL-based on-off controller. The RL-based controller is implemented as an RL agent that interacts with an epidemic simulator, implemented as an extended SEIR epidemic model. The RL agent learns a policy function that produces an optimal sequence of open/lockdown decisions such that goals specified in the RL reward function are optimized. Two concurrent goals were used: the first one is a public health goal that minimizes overshoots of ICU bed usage above an ICU bed threshold, and the second one is a socio-economic goal that minimizes the time spent under lockdowns. It is assumed that cyclic lockdowns are considered as a temporary alternative to extended lockdowns when a region faces imminent danger of overpassing resource capacity limits and when imposing an extended lockdown would cause severe social and economic consequences due to lack of necessary economic resources to support its affected population during an extended lockdown.