DeepCME: A deep learning framework for computing solution statistics of the chemical master equation.
DeepCME: A deep learning framework for computing solution statistics of the chemical master equation.
复制标题
DOI：
10.1371/journal.pcbi.1009623
复制
发表时间：
2021-12
影响因子：
4.3
通讯作者：
Khammash M
中科院分区：
文献类型：
--
作者：
Gupta A;Schwab C;Khammash M
生物分子反应网络的随机模型通常用于系统和合成生物学，以研究涉及低拷贝数物种的反应所产生的随机波动的影响。对于这样的模型，Kolmogorov的正演方程被称为化学主方程（CME），它是一个基本的线性常微分方程（ode）系统，描述了代表所有反应物种复制数的随机状态向量的概率分布的演变。这个系统的大小是由化学系统可以达到的状态数决定的，对于我们感兴趣的大多数例子来说，这个数要么很大，要么无穷大。此外，通过仅保留有限数量的重要化学状态（例如那些具有不可忽略概率的化学状态）来减小系统尺寸的近似会导致高维ODE系统，即使反应物质的数量很少。因此，精确的CME数值解是非常具有挑战性的，尽管底层的ode是线性的。人们经常求助于通过计算密集的随机模拟来估计解。本文的目标是通过使用Kolmogorov后向方程重新表述随机动力学，开发一种新的深度学习方法来计算高维cme的解统计。该方法利用深度神经网络（dnn）优越的近似特性，对状态向量的几个用户定义函数在CME解下可靠地估计期望。这种方法是基于算法的强化学习，它只需要适量的随机模拟（与典型的基于模拟的方法相比）来训练“策略函数”。这不仅允许对CME解的各种期望进行数值近似，而且允许对所有反应网络参数（例如速率常数）的灵敏度进行数值近似。我们提供了四个例子来说明我们的方法，并为未来的研究提供了几个方向。我们开发了一个深度学习框架来估计化学主方程（CME）的解，这是反应网络随机分析的基础。CME是描述随机状态向量概率密度随时间演化的常微分方程组，由于固有的维数诅咒，用现有的方法直接求解CME通常是不切实际的。此外，通常用于估计CME解决方案的基于模拟的方法通常需要大量的计算时间，即使对于中等规模的网络也是如此。为了解决这些问题，我们在本文中开发了一种基于深度强化学习的方法，称为DeepCME。DeepCME不仅基于CME解决方案估计函数期望，而且还解决了更具有挑战性的问题，即估计它们相对于所有模型参数的灵敏度。我们用四个精心挑选的不同大小的反应网络例子来说明我们的方法。我们的研究结果表明，DeepCME可靠地估计了兴趣期望值，以及所有参数灵敏度，而基于模拟的估计器的计算成本只有一小部分。我们提出了许多未来的研究方向，并提出了进一步改进DeepCME的建议，可以大大提高其准确性和适用性。
Stochastic models of biomolecular reaction networks are commonly employed in systems and synthetic biology to study the effects of stochastic fluctuations emanating from reactions involving species with low copy-numbers. For such models, the Kolmogorov’s forward equation is called the chemical master equation (CME), and it is a fundamental system of linear ordinary differential equations (ODEs) that describes the evolution of the probability distribution of the random state-vector representing the copy-numbers of all the reacting species. The size of this system is given by the number of states that are accessible by the chemical system, and for most examples of interest this number is either very large or infinite. Moreover, approximations that reduce the size of the system by retaining only a finite number of important chemical states (e.g. those with non-negligible probability) result in high-dimensional ODE systems, even when the number of reacting species is small. Consequently, accurate numerical solution of the CME is very challenging, despite the linear nature of the underlying ODEs. One often resorts to estimating the solutions via computationally intensive stochastic simulations. The goal of the present paper is to develop a novel deep-learning approach for computing solution statistics of high-dimensional CMEs by reformulating the stochastic dynamics using Kolmogorov’s backward equation. The proposed method leverages superior approximation properties of Deep Neural Networks (DNNs) to reliably estimate expectations under the CME solution for several user-defined functions of the state-vector. This method is algorithmically based on reinforcement learning and it only requires a moderate number of stochastic simulations (in comparison to typical simulation-based approaches) to train the “policy function”. This allows not just the numerical approximation of various expectations for the CME solution but also of its sensitivities with respect to all the reaction network parameters (e.g. rate constants). We provide four examples to illustrate our methodology and provide several directions for future research. We develop a deep learning framework for estimating solutions of the chemical master equation (CME) that is fundamental to stochastic analysis of reaction networks. The CME is a system of ordinary differential equations that describes the time-evolution of the probability density of the random state-vector, and owing to an inherent curse of dimensionality, directly solving the CME is generally impractical with existing approaches. Moreover, the commonly employed simulation-based approaches for estimating CME solutions often require an exorbitant amount of computational time, even for moderately-sized networks. To counter these issues, we develop a deep reinforcement learning based method, called DeepCME, in this paper. DeepCME not only estimates function expectations based on the CME solution, but it also solves the more challenging problem of estimating their sensitivities with respect to all the model parameters. We illustrate our approach with four carefully chosen reaction network examples with varying sizes. Our results demonstrate that DeepCME reliably estimates the expectations of interest, along with all the parametric sensitivities, at a fraction of the computational cost of simulation-based estimators. We present many directions for future research and suggest further improvements to DeepCME that can greatly enhance its accuracy and applicability.