Inference for Dynamic System Models

通过概率微分方程求解器量化不确定性：微分方程的数值解在数值分析领域已经得到了很好的发展，其中Runga-Kutta求解器保持了Lipschitz连续初值问题的数值解近似误差的界限。然而，超出简单的模型，数值误差变得重要和复杂。拟议的项目认为，概率解微分方程模型作为一种方式来量化数值分析的不确定性。这项工作提出了使用一个概率微分方程求解器的基础上高斯过程（GP）回归模型与非平稳协方差结构纳入模型动态估计模型的解决方案，同时量化的不确定性概率。GP回归在状态和导数函数空间中同时工作，同时递归地估计系统解。首先，状态空间中的一个点在下一个时间点被提前预测。然后，在状态和导数空间中平滑预测及其不确定性，更新预测。最后，将点和导数中的不确定性结转到下一个时间点的预测中。这项工作的应用是数值求解器的离散化会导致噪音的模型。混沌系统的特征在于两个解的发散，这两个解开始相距一定的距离，因此由求解器引起的解的不确定性的概率描述将导致目前被忽略的诚实的不确定性量化。ABC和实验设计：近似贝叶斯计算（ABC）允许在可能性难以处理或计算不可行的复杂模型上进行推理。假设模型可以针对任何参数集进行评估，则使用伪似然来比较模型和数据的汇总统计量。然后，通常使用马尔可夫链蒙特卡罗进行参数推断。推理的成功取决于找到合适的汇总统计量，理想情况下，这将是足够的统计量，尽管这很少可能或计算上可行。这个项目涉及的情况下，一些数据是可用的，但我们希望通过一个连续的实验设计，以增加新的观察。标准实验设计方法建议在设计点进行新的观察，以最小化标准，例如后验参数协方差矩阵的行列式或后验参数协方差矩阵的最大特征值等。然而，在ABC方法中，不仅存在选择设计点以改善不确定性的独特机会，而且还可以同时选择在推断中可能包括新的汇总统计量。包括新的汇总统计量允许对原始数据无用的汇总，但可以提供关于不受原始汇总统计量和/或数据集影响的新模型属性的见解，并允许将可识别性分析纳入汇总选择。此外，序贯设计方案通过序贯蒙特卡罗方法对参数估计进行了相关扩展。

Uncertainty Quantification via Probabilistic Differential Equation Solver:The numerical solution of differential equations has been well developed in the field of numerical analysis, where Runga-Kutta solvers maintain a bound on the numerical solution approximation error for Lipschitz continuous initial value problems. However, extending beyond simple models the numerical error becomes important and complex. The proposed project considers a probabilistic solution to differential equation models as a way to quantify the uncertainty of the numerical analysis. This work proposes to use a probabilistic differential equation solver based on a Gaussian Process (GP) regression model with a non-stationary covariance structure incorporating the model dynamics to estimate the model solution while quantifying uncertainty probabilistically. The GP regression works simultaneously in the state and derivative function spaces while recursively estimating the system solution. First, a point in the state space is predicted ahead at the next time point. Then, the prediction and its uncertainty are smoothed in the state and derivative spaces, updating the prediction. Finally, the uncertainty in the point and derivative are carried forward into the prediction at the next time point. Applications for this work are models where discretization of numerical solvers induces noise. Chaotic systems are characterized by the divergence of two solutions which begin some small epsilon distance apart, consequently a probabilistic description of the solution uncertainty induced by the solver will lead to honest uncertainty quantification that is currently ignored. ABC and Design of Experiments:Approximate Bayesian Computation (ABC) allows inference on complex models where the likelihood is intractable or computationally infeasible. Assuming that the model can be evaluated for any parameter set, a pseudo likelihood is used to compare summary statistics for the model and the data. Parameter inference is then generally performed using Markov Chain Monte Carlo. The success of the inference depends on finding suitable summary statistics, which ideally would be sufficient statistics, although that is rarely possible or computationally feasible. This project concerns the situation where some data is available but we wish to augment that with new observations through a sequential experimental design. Standard Design of Experiments methods suggest taking new observations at design points that minimize a criterion such as the determinant of the posterior parameter covariance matrix or the largest eigenvalue of the posterior parameter covariance matrix, etc... However in ABC methods, the unique opportunity exists to not only select the design points to improve uncertainty, but to simultaneously choose to possibly include new summary statistics in the inference. Including new summary statistics allows for summaries that were not useful with the original data but may offer insights about new model attributes unaffected by the original set of summary statistics and/or data, and allows identifiability analysis to be incorporated into summary selection. Additionally, the sequential design proposal has related extensions into parameter estimation through sequential monte carlo methods.