Convex Representation Learning for Generalized Invariance in Semi-Inner-Product Space
Convex Representation Learning for Generalized Invariance in Semi-Inner-Product Space
复制标题
DOI：
--
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Yingyi Ma;Vignesh Ganapathiraman;Yaoliang Yu;Xinhua Zhang
中科院分区：
文献类型：
--
作者：
Yingyi Ma;Vignesh Ganapathiraman;Yaoliang Yu;Xinhua Zhang
不变性（在一般意义上定义）一直是表示学习最有效的先验之一。参数模型的直接因式分解只适用于小范围的不变性，而正则化方法，尽管提高了通用性，导致非凸优化。在这项工作中，我们开发了一个凸表示学习算法的各种广义不变性，可以建模为半规范。在半内积空间中引入了新的欧氏嵌入，并建立了近似界。这使得不变表示可以有效地学习，正如我们的实验所证实的那样，沿着准确的预测。
Invariance (defined in a general sense) has been one of the most effective priors for representation learning. Direct factorization of parametric models is feasible only for a small range of invariances, while regularization approaches, despite improved generality, lead to nonconvex optimization. In this work, we develop a convex representation learning algorithm for a variety of generalized invariances that can be modeled as semi-norms. Novel Euclidean embeddings are introduced for kernel representers in a semi-inner-product space, and approximation bounds are established. This allows invariant representations to be learned efficiently and effectively as confirmed in our experiments, along with accurate predictions.