Stochastic Neighbor Embedding under f-divergences
Stochastic Neighbor Embedding under f-divergences
复制标题
f 散度下的随机邻域嵌入
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
K. Branson
中科院分区：
文献类型：
--
作者：
Daniel Jiwoong Im;Nakul Verma;K. Branson
t分布随机邻居嵌入（t-SNE）是一种强大而流行的高维数据可视化方法。它最小化了原始和嵌入数据分布之间的Kullback-Leibler （KL）散度。在这项工作中，我们建议将该方法推广到其他f-散度。我们使用原始的kl -散度和提出的f-散度泛化方法对潜在结构的类型（流形、聚类和层次）进行了分析和经验评估，并发现不同的散度对不同类型的结构表现更好。
The t-distributed Stochastic Neighbor Embedding (t-SNE) is a powerful and popular method for visualizing high-dimensional data. It minimizes the Kullback-Leibler (KL) divergence between the original and embedded data distributions. In this work, we propose extending this method to other f-divergences. We analytically and empirically evaluate the types of latent structure-manifold, cluster, and hierarchical-that are well-captured using both the original KL-divergence as well as the proposed f-divergence generalization, and find that different divergences perform better for different types of structure.
A common concern with $t$-SNE criterion is that it is optimized using gradient descent, and can become stuck in poor local minima. We propose optimizing the f-divergence based loss criteria by minimizing a variational bound. This typically performs better than optimizing the primal form, and our experiments show that it can improve upon the embedding results obtained from the original $t$-SNE criterion as well.