Generative models on manifold

在这个项目中，我们将研究流形学习假设下的生成模型。我们将特别关注扩散模型，作为生成模型的一个特例，由于其巨大的经验成功，最近在机器学习社区受到了很多关注。扩散模型现在在广泛的领域找到应用，包括图像和声音生成，医学，蛋白质设计等。在过去的三年中，在理解这些模型在实践中表现良好的原因和时间方面取得了许多进展，例如Sohl-Dickstein等人。Song et al. 2020对这种方法的早期工作和De Bortoli et al. 2021；Oko， Akiyama和Suzuki 2023仅举几例[在这里添加更多裁判]。除了Bortoli 2022和Oko， Akiyama和Suzuki 2023在某种程度上，当数据的分布属于环境空间的低维子流形时，除了流形是仿射的相对简单和不现实的场景外，对生成过程的行为知之甚少。我们将首先将Oko， Akiyama和Suzuki 2023的工作扩展到密度在具有维数d的一般未知流形上具有给定平滑度B的情况。我们将首先考虑环境空间d的维数是固定的，并研究扩散生成模型如何适应流形和密度的平滑度。然后，我们将研究高维D的更复杂的框架，即D随n增长。目的和目标我们的目标是了解在流形中训练的数据去噪扩散模型的行为，这是一个重要的应用，因为人们知道许多重要的数据集满足流形假设，例如成像数据。作为副产品，我们将深入了解扩散去噪模型的样本复杂性，这是一个重要的理论问题，在很大程度上是开放的。我们还希望深入了解神经网络如何适应数据集的几何形状；这是一个困难的问题，因此即使是微小的进展也将是非常重要的。研究方法的新颖性去噪扩散模型是一种相当新的生成模型。尽管取得了令人印象深刻的经验成功，但我们对其性质的理论理解仍然非常有限，特别是在数据生活在歧管上的重要场景中。因此，我们将要解决的问题是公开的。我们提出的渐近方法在去噪扩散模型的背景下也是新颖的；文献主要集中在非渐近界，但相对简单的情况，如仿射流形。考虑渐近边界是一种松弛，它可以让我们处理更一般的情况。与EPSRC战略和研究领域保持一致该项目属于EPSRC统计和应用概率以及人工智能技术。

Brief description of the context of the research including potential impact In this project we will study generative models under the manifold learning hypothesis. In particular we will focus on diffusion models, as a special case of generative models which have enjoyed recently much attention in the machine learning community due to their immense empirical success. Diffusion models now find applications in a wide range of fields including image and sound generation, medicine, protein design etc. Over the last three years there has been a number of advances to understand why and when these models perform well in practice, see for instance Sohl-Dickstein et al. 2015; Song et al. 2020 for the early work on such approaches and De Bortoli et al. 2021; Oko, Akiyama, and Suzuki 2023 to name but a few [add more refs here]. Apart from Bortoli 2022 and to some extent Oko, Akiyama, and Suzuki 2023, little is known on the behaviour of the generative procedure when the distribution of the data belongs to a low dimensional sub-manifold of the ambient space, except for the relatively simple and unrealistic scenario where the the manifold is affine. We will first extend the work of Oko, Akiyama, and Suzuki 2023 to the case where the density has a given smoothness B on a general, unknown manifold with dimension d. We will begin by considering that the dimension of the ambient space D is fixed and study how diffusion generative models can adapt to the manifold and to the smoothness of the density . We will then investigate the more complicated framework of high dimensional D, i.e. D grows with n. Aims and Objectives We aim to understand the behaviour of denoising diffusion models trained on data living in a manifold, an important application, since it is understood that many important datasets satisfy the manifold hypothesis, e.g. imaging data.As a by-product we will gain insight into the sample complexity of diffusion denoising models, an important theoretical questions that is largely open. We also expect to gain some insight into how neural networks adapt to the geometry of the data-set; this is a difficult question, so even modest progress will be of great importance.Novelty of the research methodology Denoising diffusion models are a fairly recent type of generative model. Despite impressive empirical success our theoretical understanding of their properties is still very limited especially in the important scenario where the data lives on the manifold. The problem we will be tackling is therefore open. The asymptotic approach we propose is also novel in the context of denoising diffusion models; the literature has focused mainly on non-asymptotic bounds but for relatively simple scenarios like affine manifolds. Considering asymptotic bounds is a relaxation which may allow us to treat much more general scenarios.Alignment to EPSRC strategies and research areas This project falls within the EPSRC Statistics and applied probability and also Artificial intelligence technologies.