Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
复制标题
DOI：
10.48550/arxiv.2310.09336
复制
发表时间：
2023-10
期刊：
影响因子：
--
通讯作者：
Maya Okawa;Ekdeep Singh Lubana;Robert P. Dick;Hidenori Tanaka
中科院分区：
文献类型：
--
作者：
Maya Okawa;Ekdeep Singh Lubana;Robert P. Dick;Hidenori Tanaka
现代生成模型显示出前所未有的生成极其逼真的数据的能力。然而，鉴于现实世界固有的组合性，这些模型在实际应用中的可靠使用要求它们具有组成一组新概念的能力，以生成训练数据集中看不到的输出。先前的工作表明，最近的扩散模型确实显示出耐人寻味的成分泛化能力，但也会出人意料地失败。受此启发，我们进行了一项对照研究，以了解条件扩散模型在合成环境中的成分概括，改变训练数据的不同属性，并测量模型生成非分布样本的能力。我们的结果表明：(I)从一个概念生成样本并合成它们的能力由潜在数据生成过程的结构决定；(Ii)由于对组成任务性能的乘法依赖，成分任务的表现表现出突然的“涌现”，部分解释了生成模型中出现的涌现现象；(Iii)与生成分布内样本相比，合成训练数据中频率较低的概念来生成分布外样本需要更多的优化步骤。总体而言，我们的研究为从以数据为中心的角度理解生成模型中的能力和组合性奠定了基础。
Modern generative models exhibit unprecedented capabilities to generate extremely realistic data. However, given the inherent compositionality of the real world, reliable use of these models in practical applications requires that they exhibit the capability to compose a novel set of concepts to generate outputs not seen in the training data set. Prior work demonstrates that recent diffusion models do exhibit intriguing compositional generalization abilities, but also fail unpredictably. Motivated by this, we perform a controlled study for understanding compositional generalization in conditional diffusion models in a synthetic setting, varying different attributes of the training data and measuring the model's ability to generate samples out-of-distribution. Our results show: (i) the order in which the ability to generate samples from a concept and compose them emerges is governed by the structure of the underlying data-generating process; (ii) performance on compositional tasks exhibits a sudden"emergence"due to multiplicative reliance on the performance of constituent tasks, partially explaining emergent phenomena seen in generative models; and (iii) composing concepts with lower frequency in the training data to generate out-of-distribution samples requires considerably more optimization steps compared to generating in-distribution samples. Overall, our study lays a foundation for understanding capabilities and compositionality in generative models from a data-centric perspective.