Reinforcement learning of rare diffusive dynamics
Reinforcement learning of rare diffusive dynamics
复制标题
稀有扩散动力学的强化学习
DOI：
10.1063/5.0057323
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Avishek Das;Dominic C. Rose;J. P. Garrahan;David T. Limmer
中科院分区：
文献类型：
--
作者：
Avishek Das;Dominic C. Rose;J. P. Garrahan;David T. Limmer
我们提出了一种直接使用强化学习来探测稀有分子动力学轨迹的方法。我们考虑的轨迹是有条件的配置空间区域之间的过渡在有限的时间内，如那些相关的反应事件的研究，和轨迹表现出罕见的波动的时间积分量在很长的时间限制，如那些相关的大偏差函数的计算。在这两种情况下，强化学习技术用于优化增加的力，使条件轨迹集合和驱动轨迹集合之间的Kullback-Leibler发散最小化。在优化的附加力下，系统将罕见的波动演化为典型的波动，从而提供了其在原始轨迹系综中的可能性的变分估计。采用低方差梯度值函数，提出了提高最优力的收敛性。我们开发的方法，采用这些梯度导致有效和准确的估计的最佳力量和各种模型系统的罕见事件的可能性。
We present a method to probe rare molecular dynamics trajectories directly using reinforcement learning. We consider trajectories that are conditioned to transition between regions of configuration space in finite time, such as those relevant in the study of reactive events, and trajectories exhibiting rare fluctuations of time-integrated quantities in the long time limit, such as those relevant in the calculation of large deviation functions. In both cases, reinforcement learning techniques are used to optimize an added force that minimizes the Kullback-Leibler divergence between the conditioned trajectory ensemble and a driven one. Under the optimized added force, the system evolves the rare fluctuation as a typical one, affording a variational estimate of its likelihood in the original trajectory ensemble. Low variance gradients employing value functions are proposed to increase the convergence of the optimal force. The method we develop employing these gradients leads to efficient and accurate estimates of both the optimal force and the likelihood of the rare event for a variety of model systems.