Inference for Dynamic System Models

*微分方程的数值解在数值分析领域得到了很好的发展，其中Runga-Kutta解保持了Lipschitz连续初值问题数值解近似误差的一个界。然而，扩展到简单模型之外，数值误差变得重要而复杂。提出的项目考虑微分方程模型的概率解作为量化数值分析的不确定性的一种方法。本文提出了一种基于高斯过程（GP）回归模型的概率微分方程求解器，该模型具有非平稳协方差结构，结合模型动力学来估计模型解，同时对不确定性进行概率量化。在递归估计系统解的同时，GP回归在状态和导数函数空间中同时工作。首先，在下一个时间点提前预测状态空间中的一个点。然后在状态空间和导数空间中对预测结果及其不确定性进行平滑处理，更新预测结果。最后，将该点及其导数的不确定性结转到下一个时间点的预测中。**本工作的应用是数值解算的离散化引起噪声的模型。混沌系统的特点是两个解的散度开始时距离很小，因此由求解器引起的解不确定性的概率描述将导致目前被忽视的诚实的不确定性量化。**近似贝叶斯计算（ABC）允许在可能性难以处理或计算上不可行的复杂模型上进行推理。假设模型可以对任何参数集进行评估，则使用伪似然来比较模型和数据的汇总统计。然后一般使用马尔可夫链蒙特卡罗进行参数推理。推理的成功取决于找到合适的汇总统计数据，理想情况下，这将是充分的统计数据，尽管这很少是可能的或计算上可行的。这个项目涉及的情况是，一些数据是可用的，但我们希望通过一个连续的实验设计来增加新的观察结果。标准实验设计方法建议在设计点进行新的观察，使后验参数协方差矩阵的行列式或后验参数协方差矩阵的最大特征值等标准最小化。然而，在ABC方法中，独特的机会不仅存在于选择设计点以改善不确定性，而且同时选择在推理中可能包括新的汇总统计。包括新的汇总统计数据允许对原始数据无用的汇总，但可以提供对不受原始汇总统计数据集和/或数据影响的新模型属性的见解，并允许将可识别性分析合并到汇总选择中。此外，序贯设计方案通过序贯蒙特卡罗方法扩展到参数估计。

Uncertainty Quantification via Probabilistic Differential Equation Solver:*The numerical solution of differential equations has been well developed in the field of numerical analysis, where Runga-Kutta solvers maintain a bound on the numerical solution approximation error for Lipschitz continuous initial value problems. However, extending beyond simple models the numerical error becomes important and complex. The proposed project considers a probabilistic solution to differential equation models as a way to quantify the uncertainty of the numerical analysis. This work proposes to use a probabilistic differential equation solver based on a Gaussian Process (GP) regression model with a non-stationary covariance structure incorporating the model dynamics to estimate the model solution while quantifying uncertainty probabilistically. The GP regression works simultaneously in the state and derivative function spaces while recursively estimating the system solution. First, a point in the state space is predicted ahead at the next time point. Then, the prediction and its uncertainty are smoothed in the state and derivative spaces, updating the prediction. Finally, the uncertainty in the point and derivative are carried forward into the prediction at the next time point. **Applications for this work are models where discretization of numerical solvers induces noise. Chaotic systems are characterized by the divergence of two solutions which begin some small epsilon distance apart, consequently a probabilistic description of the solution uncertainty induced by the solver will lead to honest uncertainty quantification that is currently ignored. **ABC and Design of Experiments:*Approximate Bayesian Computation (ABC) allows inference on complex models where the likelihood is intractable or computationally infeasible. Assuming that the model can be evaluated for any parameter set, a pseudo likelihood is used to compare summary statistics for the model and the data. Parameter inference is then generally performed using Markov Chain Monte Carlo. The success of the inference depends on finding suitable summary statistics, which ideally would be sufficient statistics, although that is rarely possible or computationally feasible. This project concerns the situation where some data is available but we wish to augment that with new observations through a sequential experimental design. Standard Design of Experiments methods suggest taking new observations at design points that minimize a criterion such as the determinant of the posterior parameter covariance matrix or the largest eigenvalue of the posterior parameter covariance matrix, etc... However in ABC methods, the unique opportunity exists to not only select the design points to improve uncertainty, but to simultaneously choose to possibly include new summary statistics in the inference. Including new summary statistics allows for summaries that were not useful with the original data but may offer insights about new model attributes unaffected by the original set of summary statistics and/or data, and allows identifiability analysis to be incorporated into summary selection. Additionally, the sequential design proposal has related extensions into parameter estimation through sequential monte carlo methods.