CARD: Classification and Regression Diffusion Models

CARD: Classification and Regression Diffusion Models
复制标题

DOI：
10.48550/arxiv.2206.07275
复制
发表时间：
2022-06
期刊：
ArXiv
影响因子：
--
通讯作者：
Xizewen Han;Huangjie Zheng;Mingyuan Zhou
Xizewen Han;Huangjie Zheng;Mingyuan Zhou
中科院分区：
其他
文献类型：
--
作者：
Xizewen Han;Huangjie Zheng;Mingyuan Zhou

文献摘要

相似文献

在给定协变量 $\boldsymbol x$ 的情况下学习连续或分类响应变量 $\boldsymbol y$ 的分布是统计学和机器学习中的基本问题。基于深度神经网络的监督学习算法在预测给定 $\boldsymbol x$ 的 $\boldsymbol y$ 均值方面取得了巨大进步，但它们经常因其准确捕捉预测不确定性的能力而受到批评。在本文中，我们引入分类和回归扩散（CARD）模型，该模型结合了基于去噪扩散的条件生成模型和预训练的条件均值估计器，可以在给定$\boldsymbol x$的情况下准确预测$\boldsymbol y$的分布。我们通过玩具示例和真实数据集证明了 CARD 在条件分布预测方面的出色能力，实验结果表明 CARD 一般优于最先进的方法，包括为不确定性估计而设计的基于贝叶斯神经网络的方法，特别是当给定 $\boldsymbol x$ 的 $\boldsymbol y$ 的条件分布是多模态时。此外，我们利用生成模型输出的随机性，在分类任务的实例级别获得更细粒度的模型置信度评估。
Learning the distribution of a continuous or categorical response variable $\boldsymbol y$ given its covariates $\boldsymbol x$ is a fundamental problem in statistics and machine learning. Deep neural network-based supervised learning algorithms have made great progress in predicting the mean of $\boldsymbol y$ given $\boldsymbol x$, but they are often criticized for their ability to accurately capture the uncertainty of their predictions. In this paper, we introduce classification and regression diffusion (CARD) models, which combine a denoising diffusion-based conditional generative model and a pre-trained conditional mean estimator, to accurately predict the distribution of $\boldsymbol y$ given $\boldsymbol x$. We demonstrate the outstanding ability of CARD in conditional distribution prediction with both toy examples and real-world datasets, the experimental results on which show that CARD in general outperforms state-of-the-art methods, including Bayesian neural network-based ones that are designed for uncertainty estimation, especially when the conditional distribution of $\boldsymbol y$ given $\boldsymbol x$ is multi-modal. In addition, we utilize the stochastic nature of the generative model outputs to obtain a finer granularity in model confidence assessment at the instance level for classification tasks.