Simulation-based Methods for Large Dynamic Latent Variable Models with Unobserved Heterogeneity

在科学和工程中，人们经常需要通过潜变量模型来研究复杂动态系统的行为。大多数情况下，所研究的系统都很大。例如，地球科学中的全球尺度数值天气预报系统需要估计超过10^9个状态变量，每隔几个小时就有大约10^7个观测值。另一个例子是一个网络形成模型，它包含大量的参数，这些参数可能会随着数据的复杂性而缓慢增加，这是未观察到的异质性的直接结果，这种现象是研究中的单元具有未观察到的个体特定特征，可能会影响它们的行为。因此，目前的研究工作应该集中在创建新的方法，具有高度的可扩展性，以估计这些大规模的潜变量模型。在这个建议中，我提出了一个新的研究议程，在这个方向上，这是建立在我最近的工作在时间序列和面板数据计量经济学。 形式上，潜变量模型是通过观测过程和未观测马尔可夫过程的联合概率密度函数（pdf）定义的，并且该pdf可能不具有易于处理的数学表达式。这些过程取决于需要估计的[未知]参数。如果数据具有低维度并且参数的数量不随着数据量的增加而增加，则使用从经典或贝叶斯角度开发的最新技术，该估计任务可能是可行的。这些技术分为三类：a）间接推理; B）数据克隆、迭代过滤和使用贝叶斯映射的迭代过滤（用于最大化难以处理的 似然函数）;以及c）用于计算后验统计对象（诸如后验均值/模式和概率密度）的近似贝叶斯计算（ABC）。为了规避现有方法所面临的不可扩展性问题，我提出了基于模拟的方法，承诺一个快速，准确的方法来估计和验证大规模的动态潜变量模型。 具有时间序列/面板和网络数据的潜变量模型的应用在许多领域中无处不在，包括计量经济学、生物学、生态学、流行病学、神经科学、信号处理和各个领域。事实上，许多上述的统计方法来计算模型的最大似然估计没有听话的似然函数，首先提出了解决生物学，气候学和生态学的科学问题。因此，所提出的方法不限于计量经济学/统计学应用。事实上，这些方法是基于非常一般的动态模型推导出来的，允许读者充分利用每种技术来满足其特定的应用。

In science and engineering, one often needs to study the behaviour of complex dynamic systems through latent variable models. Most of the time, the systems under study are large. For example, a global-scale numerical weather prediction system in geoscience requires estimation of over 10^9 state variables given about 10^7 observations every few hours. Another example is a network-formation model which contains a large number of parameters which may slowly increase with the data complexity as a direct consequence of unobserved heterogeneity a phenomenon where units under study have unobserved individual-specific characteristics that may affect their behaviors. Therefore, current research endeavours should be focused on creating new methods with high scalability to estimate these large-scale latent variable models. In this proposal, I propose a new research agenda in this direction, which builds upon my recent work in time series and panel data econometrics. Formally, a latent variable model is defined via the joint probability density function (pdf) of an observation process and an unobserved Markov process and this pdf may not possess a tractable mathematical expression. The processes depend on [unknown] parameters that need to be estimated. This estimation task may be feasible using recent techniques developed from the classical or Bayesian perspective if the data has low dimension and the number of parameters does not increase with the amount of data. These techniques are classified into three categories: a) indirect inference; b) data cloning, iterated filtering, and iterated filtering with Bayes maps (used to maximize intractable likelihood functions); and c) approximate Bayesian computation (ABC) used to compute posterior statistical objects (such as posterior means/modes and probability densities). To circumvent the unscalability problems faced by the existing approaches, I propose simulation-based methods that promise a fast and accurate way to estimate and validate large-scale dynamic latent variable models. Applications of latent variable models with time series/panel and network data are ubiquitous in many areas, including econometrics, biology, ecology, epidemiology, neuroscience, signal processing, and various fields. In fact, many aforementioned statistical methods to compute maximum likelihood estimates for models without tractable likelihood functions were first proposed to address scientific problems in biology, climatology, and ecology. Therefore, the proposed methods are not limited to econometric/statistical applications. In fact, the methods are derived based on very general dynamical models, allowing the readers to make the most of each technique for their specific applications.