Generative models on manifold

简要说明研究的背景，包括在本项目中的潜在影响，我们将在流形学习假设下研究生成模型。特别是，我们将重点介绍扩散模型，作为生成模型的特例，由于其巨大的经验成功，生成模型最近在机器学习界受到了极大的关注。扩散模型现在被广泛应用于图像和声音生成、医学、蛋白质设计等领域。在过去的三年中，在理解为什么以及何时这些模型在实践中表现良好方面取得了许多进展，例如参见Sohl-Dickstein等人。2015年；Song等人。关于这种方法的早期工作，以及de Bortoli等人。2021；Oko，Akiyama，Suzuki 2023，仅举几个[在此添加更多参考文献]。除了Bortoli 2022以及一定程度上Oko、Akiyama和Suzuki 2023之外，除了相对简单和不现实的流形是仿射的情况外，对于数据分布属于环境空间的低维子流形时的生成过程的行为知之甚少。我们首先将Oko，Akiyama和Suzuki 2023的工作推广到一般未知流形上密度具有给定光滑性B的情形。我们将首先考虑环境空间D的维度是固定的，并研究扩散生成模型如何适应流形和密度的光滑性。然后我们将研究高维D的更复杂的框架，即D与n一起增长。目的和目标我们的目标是理解基于流形中的数据训练的去噪扩散模型的行为，这是一个重要的应用，因为人们理解许多重要的数据集满足流形假设，例如成像数据。作为副产品，我们将深入了解扩散去噪模型的样本复杂性，这是一个在很大程度上开放的重要理论问题。我们还希望深入了解神经网络如何适应数据集的几何结构；这是一个困难的问题，因此即使是微小的进展也将是非常重要的。去噪扩散模型的研究方法是一种相当新的生成性模型。尽管取得了令人印象深刻的经验成果，但我们对它们的性质的理论理解仍然非常有限，特别是在数据位于流形上的重要场景中。因此，我们将要解决的问题是开放的。我们提出的渐近方法在去噪扩散模型的背景下也是新颖的；文献主要集中在非渐近界，但对于相对简单的场景，如仿射流形。考虑渐近边界是一种放松，这可能允许我们处理更一般的情况。与EPSRC的策略和研究领域保持一致，这个项目属于EPSRC统计和应用概率以及人工智能技术。

Brief description of the context of the research including potential impact In this project we will study generative models under the manifold learning hypothesis. In particular we will focus on diffusion models, as a special case of generative models which have enjoyed recently much attention in the machine learning community due to their immense empirical success. Diffusion models now find applications in a wide range of fields including image and sound generation, medicine, protein design etc. Over the last three years there has been a number of advances to understand why and when these models perform well in practice, see for instance Sohl-Dickstein et al. 2015; Song et al. 2020 for the early work on such approaches and De Bortoli et al. 2021; Oko, Akiyama, and Suzuki 2023 to name but a few [add more refs here]. Apart from Bortoli 2022 and to some extent Oko, Akiyama, and Suzuki 2023, little is known on the behaviour of the generative procedure when the distribution of the data belongs to a low dimensional sub-manifold of the ambient space, except for the relatively simple and unrealistic scenario where the the manifold is affine. We will first extend the work of Oko, Akiyama, and Suzuki 2023 to the case where the density has a given smoothness B on a general, unknown manifold with dimension d. We will begin by considering that the dimension of the ambient space D is fixed and study how diffusion generative models can adapt to the manifold and to the smoothness of the density . We will then investigate the more complicated framework of high dimensional D, i.e. D grows with n. Aims and Objectives We aim to understand the behaviour of denoising diffusion models trained on data living in a manifold, an important application, since it is understood that many important datasets satisfy the manifold hypothesis, e.g. imaging data.As a by-product we will gain insight into the sample complexity of diffusion denoising models, an important theoretical questions that is largely open. We also expect to gain some insight into how neural networks adapt to the geometry of the data-set; this is a difficult question, so even modest progress will be of great importance.Novelty of the research methodology Denoising diffusion models are a fairly recent type of generative model. Despite impressive empirical success our theoretical understanding of their properties is still very limited especially in the important scenario where the data lives on the manifold. The problem we will be tackling is therefore open. The asymptotic approach we propose is also novel in the context of denoising diffusion models; the literature has focused mainly on non-asymptotic bounds but for relatively simple scenarios like affine manifolds. Considering asymptotic bounds is a relaxation which may allow us to treat much more general scenarios.Alignment to EPSRC strategies and research areas This project falls within the EPSRC Statistics and applied probability and also Artificial intelligence technologies.