Simulation and Inference Algorithms for Stochastic Differential Equations

在神经科学、生物化学、分子和细胞生物学、统计物理学和金融学等领域，许多研究人员投入时间建立第一性原理数学模型。随机微分方程（SDE）是这些模型的一个子集，自然地解释了非线性和噪声的综合效应，这些效应在许多当代感兴趣的系统中观察到。这种模型通常有大量不能直接观测到的参数。研究人员从他们正在研究的系统中收集数据，并试图利用这些数据来推断模型中的参数。另外，研究人员从一个特定系统的数据开始，通常希望预测系统的未来行为，特别是在科学背景下，解释为什么系统会这样做。在这个项目中，PI将开发算法、方法和开源软件包，显著提高研究人员在随机模型中推断参数的能力，并开发具有强大预测和/或解释力的随机模型。新方法将自然地适应具有一种或多种病态的数据，例如不规则的观察时间，大体积和非高斯（即重尾）特征。推理问题的关键是计算与SDE模型的时间序列观测相关联的似然函数。先前的工作表明，在此推理问题的确定性方法中，基于Fokker-Planck解算器的方法是最准确的。对于由标准布朗运动驱动的SDE，在相同的误差水平下，用正交（DTQ）进行密度跟踪计算的可能性比可比的福克-普朗克求解器快几个数量级。在此基础上，本工作寻求开发比现有技术更准确、更高效和可扩展的基于dtq的推理方法。PI将把DTQ推广到levy驱动的SDE，通过高阶近似提高DTQ的效率，设计可扩展的伴随方法来计算似然梯度，并应用这些伴随方法来最大化对数似然和对数后验。PI还将对SDE推断技术进行全面比较，包括通过期望最大化和变分贝叶斯等方法进行近似推断。该项目的一个关键部分是以开源包的形式开发软件基础设施，以便与R、Python和Apache Spark等框架一起使用。这些软件包将使所有感兴趣的研究人员，特别是那些没有受过高级随机学培训的研究人员，能够利用基于dtq的模拟和推理方法。在这个项目中开发的所有新代码都将被彻底记录和测试。pi还计划将开发的软件作为开源发布，并通过确保感兴趣的研究人员能够为开发的软件的代码库做出贡献，从而围绕该语言建立一个用户社区。这将使该项目得到更广泛的发展。高级网络基础设施办公室的软件集群对这方面特别感兴趣，该办公室为该奖项提供了共同资金。

In fields such as neuroscience, biochemistry, molecular and cellular biology, statistical physics and finance, many researchers have invested time to build first-principle mathematical models. Stochastic differential equations (SDE), a subset of such models, naturally account for the combined effects of nonlinearity and noise, effects observed in many systems of contemporary interest. Such models often have large numbers of parameters that cannot be directly observed. Researchers collect data from the system they are studying and seek to use this data to infer parameters in their models. Alternatively, researchers who begin with data from a particular system typically desire forecasts regarding the future behavior of the system and, especially in scientific contexts, explanations as to why the system behaves the way that it does. In this project, the PI will develop algorithms, methods, and open-source software packages that significantly improve a researcher's ability to infer parameters in stochastic models, and to develop stochastic models with strong predictive and/or explanatory power. The new methods will naturally accommodate data with one or more pathologies, such as irregular observation times, large volume, and non-Gaussian (i.e., heavy-tailed) features.The crux of the inference problem is the computation of the likelihood function associated with time series observations of an SDE model. Prior work has shown that, among deterministic methods for this inference problem, methods based on Fokker-Planck solvers are the most accurate. For SDE driven by standard Brownian motion, density tracking by quadrature (DTQ) computes likelihoods orders of magnitude faster than comparable Fokker-Planck solvers for the same level of error. Building on this foundation, this work seeks to develop DTQ-based inference methods that are more accurate, efficient and scalable than existing techniques. The PI will generalize DTQ to Levy-driven SDE, increase DTQ's efficiency through higher-order approximations, design scalable adjoint methods to compute gradients of the likelihood, and apply these adjoint methods to maximize log likelihoods and log posteriors. The PI will also conduct a thorough comparison of SDE inference techniques, including approximate inference through methods such as expectation maximization and variational Bayes. A key part of the project is the development of software infrastructure in the form of open-source packages for use with frameworks such as R, Python, and Apache Spark. These packages will enable all interested researchers, especially those with no advanced training in stochastics, to make use of DTQ-based simulation and inference methods. All new codes developed in this project will be thoroughly documented and tested. The PIs also plan to release the software developed as open source and build a user community around the language by ensuring that interested researchers are able to contribute to the codebase of the software developed. This will allow a wider growth of the project. This aspect is of special interest to the software cluster in the Office of Advanced Cyberinfrastructure, which has provided co-funding for this award.