Simulation-based Methods for Large Dynamic Latent Variable Models with Unobserved Heterogeneity

在科学和工程中，人们经常需要通过潜在变量模型来研究复杂动态系统的行为。大多数时候，所研究的系统都很大。例如，地球科学中的全球尺度数值天气预报系统需要估计超过10^9个状态变量，每隔几小时进行大约10^7次观测。另一个例子是包含大量参数的网络形成模型，这些参数可能会随着数据复杂性的增加而缓慢增加，这是未被观察到的异质性的直接后果——一种被研究单位具有未被观察到的可能影响其行为的个体特定特征的现象。因此，当前的研究工作应侧重于创造具有高可扩展性的新方法来估计这些大规模潜在变量模型。在本提案中，我提出了一个新的研究议程，在这个方向上，它建立在我最近的工作在时间序列和面板数据计量经济学。从形式上看，潜变量模型是通过观测过程和未观测马尔可夫过程的联合概率密度函数（pdf）来定义的，这个pdf可能不具有易于处理的数学表达式。这些过程依赖于需要估计的[未知]参数。如果数据是低维的，并且参数的数量不随数据量的增加而增加，那么使用从经典或贝叶斯角度开发的最新技术，这种估计任务可能是可行的。这些技术分为三类：a)间接推理；b)数据克隆、迭代滤波和贝叶斯映射迭代滤波（用于最大化难处理似然函数）；c)近似贝叶斯计算（ABC），用于计算后验统计对象（如后验均值/模态和概率密度）。为了规避现有方法所面临的不可扩展性问题，我提出了基于仿真的方法，它保证了一种快速准确的方法来估计和验证大规模动态潜在变量模型。时间序列/面板和网络数据的潜在变量模型在计量经济学、生物学、生态学、流行病学、神经科学、信号处理等各个领域都有广泛的应用。事实上，前面提到的许多计算模型最大似然估计的统计方法，最初是为了解决生物学、气候学和生态学中的科学问题而提出的。因此，所提出的方法并不局限于计量经济学/统计学应用。事实上，这些方法都是基于非常通用的动态模型推导出来的，允许读者充分利用每种技术的特定应用。

In science and engineering, one often needs to study the behaviour of complex dynamic systems through latent variable models. Most of the time, the systems under study are large. For example, a global-scale numerical weather prediction system in geoscience requires estimation of over 10^9 state variables given about 10^7 observations every few hours. Another example is a network-formation model which contains a large number of parameters which may slowly increase with the data complexity as a direct consequence of unobserved heterogeneity - a phenomenon where units under study have unobserved individual-specific characteristics that may affect their behaviors. Therefore, current research endeavours should be focused on creating new methods with high scalability to estimate these large-scale latent variable models. In this proposal, I propose a new research agenda in this direction, which builds upon my recent work in time series and panel data econometrics. Formally, a latent variable model is defined via the joint probability density function (pdf) of an observation process and an unobserved Markov process and this pdf may not possess a tractable mathematical expression. The processes depend on [unknown] parameters that need to be estimated. This estimation task may be feasible using recent techniques developed from the classical or Bayesian perspective if the data has low dimension and the number of parameters does not increase with the amount of data. These techniques are classified into three categories: a) indirect inference; b) data cloning, iterated filtering, and iterated filtering with Bayes maps (used to maximize intractable likelihood functions); and c) approximate Bayesian computation (ABC) used to compute posterior statistical objects (such as posterior means/modes and probability densities). To circumvent the unscalability problems faced by the existing approaches, I propose simulation-based methods that promise a fast and accurate way to estimate and validate large-scale dynamic latent variable models. Applications of latent variable models with time series/panel and network data are ubiquitous in many areas, including econometrics, biology, ecology, epidemiology, neuroscience, signal processing, and various fields. In fact, many aforementioned statistical methods to compute maximum likelihood estimates for models without tractable likelihood functions were first proposed to address scientific problems in biology, climatology, and ecology. Therefore, the proposed methods are not limited to econometric/statistical applications. In fact, the methods are derived based on very general dynamical models, allowing the readers to make the most of each technique for their specific applications.