A likelihood approach to nonparametric estimation of a singular distribution using deep generative models
A likelihood approach to nonparametric estimation of a singular distribution using deep generative models
复制标题
DOI：
--
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Minwoo Chae;Dongha Kim;Yongdai Kim;Lizhen Lin
中科院分区：
文献类型：
--
作者：
Minwoo Chae;Dongha Kim;Yongdai Kim;Lizhen Lin
我们研究了使用深度生成模型对奇异分布进行非参数估计的似然方法的统计特性。更具体地说，深度生成模型用于对被假设为集中在一些低维结构周围的高维数据进行建模。估计这种低维结构（如低维流形）上支持的分布是具有挑战性的，因为它相对于环境空间中的勒贝格测度具有奇异性。在所考虑的模型中，由于奇异性，通常的似然方法可能无法一致地估计目标分布。我们证明了一个新的和有效的解决方案存在扰动的数据与实例噪声，这导致一致的估计的基础分布与理想的收敛速度。我们还描述了可以通过深度生成模型有效估计的分布类。这个类是足够普遍的，以包含各种结构化分布，如产品分布，经典光滑分布和分布上的低维流形支持。我们的分析提供了一些见解，如何深入生成模型可以避免非参数分布估计的维数灾难。我们进行了全面的仿真研究和真实的数据分析，实证证明，所提出的数据扰动技术，提高了估计性能显着。
We investigate statistical properties of a likelihood approach to nonparametric estimation of a singular distribution using deep generative models. More specifically, a deep generative model is used to model high-dimensional data that are assumed to concentrate around some low-dimensional structure. Estimating the distribution supported on this low-dimensional structure, such as a low-dimensional manifold, is challenging due to its singularity with respect to the Lebesgue measure in the ambient space. In the considered model, a usual likelihood approach can fail to estimate the target distribution consistently due to the singularity. We prove that a novel and effective solution exists by perturbing the data with an instance noise, which leads to consistent estimation of the underlying distribution with desirable convergence rates. We also characterize the class of distributions that can be efficiently estimated via deep generative models. This class is sufficiently general to contain various structured distributions such as product distributions, classically smooth distributions and distributions supported on a low-dimensional manifold. Our analysis provides some insights on how deep generative models can avoid the curse of dimensionality for nonparametric distribution estimation. We conduct a thorough simulation study and real data analysis to empirically demonstrate that the proposed data perturbation technique improves the estimation performance significantly.