Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions
Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions
复制标题
DOI：
10.48550/arxiv.2209.11215
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Sitan Chen;Sinho Chewi;Jungshian Li;Yuanzhi Li;A. Salim;Anru R. Zhang
中科院分区：
文献类型：
--
作者：
Sitan Chen;Sinho Chewi;Jungshian Li;Yuanzhi Li;A. Salim;Anru R. Zhang
我们提供了理论上的收敛保证分数为基础的生成模型（SGM），如去噪扩散概率模型（DDPMs），这构成了大规模的现实世界的生成模型，如DALL$\cdot$E 2的骨干。我们的主要结果是，假设准确的分数估计，这样的SGMs可以有效地从基本上任何现实的数据分布样本。与以前的工作相比，我们的结果（1）保持了$L^2$-准确的分数估计（而不是$L^\infty$-准确的）;（2）不需要排除大量非对数的限制性函数不等式条件;（3）在所有相关问题参数中多项式地缩放;以及（4）匹配Langevin扩散的离散化的现有技术的复杂性保证，只要分数误差足够小。我们认为，这是强有力的理论理由的经验成功的SGMs。我们还研究了基于临界阻尼朗之万扩散（CLD）的SGMs。与传统观点相反，我们提供的证据表明，CLD的使用不会降低SGMs的复杂性。
We provide theoretical convergence guarantees for score-based generative models (SGMs) such as denoising diffusion probabilistic models (DDPMs), which constitute the backbone of large-scale real-world generative models such as DALL$\cdot$E 2. Our main result is that, assuming accurate score estimates, such SGMs can efficiently sample from essentially any realistic data distribution. In contrast to prior works, our results (1) hold for an $L^2$-accurate score estimate (rather than $L^\infty$-accurate); (2) do not require restrictive functional inequality conditions that preclude substantial non-log-concavity; (3) scale polynomially in all relevant problem parameters; and (4) match state-of-the-art complexity guarantees for discretization of the Langevin diffusion, provided that the score error is sufficiently small. We view this as strong theoretical justification for the empirical success of SGMs. We also examine SGMs based on the critically damped Langevin diffusion (CLD). Contrary to conventional wisdom, we provide evidence that the use of the CLD does not reduce the complexity of SGMs.