Representing Deep Neural Networks Latent Space Geometries with Graphs
Representing Deep Neural Networks Latent Space Geometries with Graphs
复制标题
DOI：
10.3390/a14020039
复制
发表时间：
2021-02-01
期刊：
影响因子：
2.3
通讯作者：
Ortega, Antonio
中科院分区：
文献类型：
--
作者：
Lassance, Carlos;Gripon, Vincent;Ortega, Antonio
深度学习（DL）因其在许多机器学习任务中达到最先进性能的能力而引起了广泛关注。DL方法的核心原理包括以端到端的方式训练复合架构，其中输入与经过训练以优化目标函数的输出相关联。由于它们的组合性质，DL架构自然会表现出输入的几个中间表示，这些中间表示属于所谓的潜在空间。当单独对待时，这些中间表示在学习过程中大多数时候是不受约束的，因为不清楚哪些属性应该受到青睐。然而，当同时处理一批输入时，相应的一组中间表示会显示出关系（我们称之为几何），可以在这些关系上寻找所需的属性。在这项工作中，我们表明，它是可能的，这些潜在的几何形状，以解决各种问题的限制。更详细地说，我们建议通过从处理一批输入时获得的中间表示构建相似性图来表示几何形状。通过约束这些潜在几何图形（LGG），我们解决了以下三个问题：（i）通过模仿其几何形状来再现教师架构的行为，（ii）通过针对特定几何形状来设计用于分类的有效嵌入，以及（iii）通过强制连续潜在空间之间的几何形状的平滑变化来实现对输入偏差的鲁棒性。使用标准的视觉基准，我们证明了所提出的基于几何的方法在解决所考虑的问题的能力。
Deep Learning (DL) has attracted a lot of attention for its ability to reach state-of-the-art performance in many machine learning tasks. The core principle of DL methods consists of training composite architectures in an end-to-end fashion, where inputs are associated with outputs trained to optimize an objective function. Because of their compositional nature, DL architectures naturally exhibit several intermediate representations of the inputs, which belong to so-called latent spaces. When treated individually, these intermediate representations are most of the time unconstrained during the learning process, as it is unclear which properties should be favored. However, when processing a batch of inputs concurrently, the corresponding set of intermediate representations exhibit relations (what we call a geometry) on which desired properties can be sought. In this work, we show that it is possible to introduce constraints on these latent geometries to address various problems. In more detail, we propose to represent geometries by constructing similarity graphs from the intermediate representations obtained when processing a batch of inputs. By constraining these Latent Geometry Graphs (LGGs), we address the three following problems: (i) reproducing the behavior of a teacher architecture is achieved by mimicking its geometry, (ii) designing efficient embeddings for classification is achieved by targeting specific geometries, and (iii) robustness to deviations on inputs is achieved via enforcing smooth variation of geometry between consecutive latent spaces. Using standard vision benchmarks, we demonstrate the ability of the proposed geometry-based methods in solving the considered problems.