Learning finite-dimensional coding schemes with nonlinear reconstruction maps
Learning finite-dimensional coding schemes with nonlinear reconstruction maps
复制标题
DOI：
10.1137/18m1234461
复制
发表时间：
2018-12
期刊：
影响因子：
--
通讯作者：
Jaeho Lee;M. Raginsky
中科院分区：
文献类型：
--
作者：
Jaeho Lee;M. Raginsky
本文将有限维有损编码方案的Maurer-Pontil框架推广到高维随机向量映射到低维欧氏空间中的隐表示紧集的元素，并且重构映射属于给定的一类非线性映射的情况。在这种设置下，它包含了广泛的一类无监督表示学习问题，我们使用最优运输理论的工具在结构约束下建立了近似生成建模的连接。接下来，我们考虑在有限的训练样本集合的基础上学习编码方案的问题，并提出具有高概率的泛化界。然后，我们在由深度神经网络实现重建映射的环境中说明了一般理论。
This paper generalizes the Maurer--Pontil framework of finite-dimensional lossy coding schemes to the setting where a high-dimensional random vector is mapped to an element of a compact set of latent representations in a lower-dimensional Euclidean space, and the reconstruction map belongs to a given class of nonlinear maps. Under this setup, which encompasses a broad class of unsupervised representation learning problems, we establish a connection to approximate generative modeling under structural constraints using the tools from the theory of optimal transportation. Next, we consider problem of learning a coding scheme on the basis of a finite collection of training samples and present generalization bounds that hold with high probability. We then illustrate the general theory in the setting where the reconstruction maps are implemented by deep neural nets.