Evaluating generative networks using Gaussian mixtures of image features
Evaluating generative networks using Gaussian mixtures of image features
复制标题
DOI：
10.1109/wacv56688.2023.00036
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
L. Luzi;Carlos Ortiz Marrero;Nile Wynar;Richard Baraniuk;Michael J. Henry
中科院分区：
文献类型：
--
作者：
L. Luzi;Carlos Ortiz Marrero;Nile Wynar;Richard Baraniuk;Michael J. Henry
我们开发了一种用于评估生成网络性能的方法，给出了两组图像。目前用于此的流行性能度量是Fréchet初始距离（FID）。FID假设使用Inception-v3的倒数第二层特征化的图像遵循高斯分布，如果我们希望使用FID作为度量，则不能违反这一假设。然而，我们发现ImageNet数据集的Inception-v3特征不是高斯的;特别是，每个边缘都不是高斯的。为了解决这个问题，我们使用高斯混合模型（GMM）对特征图像进行建模，并计算限制于GMM的2-Wasserstein距离。我们定义了一个性能指标，我们称之为WaM，在两组图像上使用Inception-v3（或另一个分类器）来特征化图像，估计两个GARCH，并使用限制的2-Wasserstein距离来比较GARCH。我们的实验表明的优势，瓦姆FID，包括FID是如何更敏感的比瓦姆难以察觉的图像扰动。通过将从Inception-v3获得的非高斯特征建模为GMM并使用GMM度量，我们可以更准确地评估生成网络的性能。
We develop a measure for evaluating the performance of generative networks given two sets of images. A popular performance measure currently used to do this is the Fréchet Inception Distance (FID). FID assumes that images featurized using the penultimate layer of Inception-v3 follow a Gaussian distribution, an assumption which cannot be violated if we wish to use FID as a metric. However, we show that Inception-v3 features of the ImageNet dataset are not Gaussian; in particular, every single marginal is not Gaussian. To remedy this problem, we model the featurized images using Gaussian mixture models (GMMs) and compute the 2-Wasserstein distance restricted to GMMs. We define a performance measure, which we call WaM, on two sets of images by using Inception-v3 (or another classifier) to featurize the images, estimate two GMMs, and use the restricted 2-Wasserstein distance to compare the GMMs. We experimentally show the advantages of WaM over FID, including how FID is more sensitive than WaM to imperceptible image perturbations. By modelling the non-Gaussian features obtained from Inception-v3 as GMMs and using a GMM metric, we can more accurately evaluate generative network performance.