Embedding Learning
Embedding Learning
复制标题
DOI：
10.1080/01621459.2020.1775614
复制
发表时间：
2020-07-20
影响因子：
3.7
通讯作者：
Wang, Junhui
中科院分区：
文献类型：
--
作者：
Dai, Ben;Shen, Xiaotong;Wang, Junhui
数值嵌入已经成为处理和分析无法以预定义方式表示的非结构化数据的标准技术。它通过将数据映射到数字向量来存储数据的主要特征。嵌入通常是无监督的，并通过从大规模未注释数据中进行迁移学习来构建。给定嵌入，被称为两阶段方法的下游学习方法适用于非结构化数据。在本文中，我们介绍了一种新型的嵌入学习框架，以提供比两阶段方法更高的学习准确性，同时确定最佳的学习自适应嵌入。特别是，我们提出了一个U-最小的充分学习自适应嵌入的概念，在此基础上，我们寻求一个最佳的，以最大限度地提高学习精度的嵌入约束。此外，当专门的一般框架分类，我们得到一个图嵌入分类器的基础上的超链接张量表示多个超图，有向或无向，表征多路关系的非结构化数据。数值上，我们设计算法的基础上分块坐标下降和投影梯度下降实现线性和前馈神经网络分类器，分别。在理论上，我们建立了一个学习理论来量化所提出的方法的泛化误差。此外，我们表明，在线性回归中，独热编码器是更可取的两阶段的方法，但其尺寸限制阻碍了其预测性能。对于图嵌入分类器，泛化误差匹配到标准快速率或线性或非线性分类的参数率。最后，我们展示了实用的分类器在语法分类和情感分析的两个基准。
Numerical embedding has become one standard technique for processing and analyzing unstructured data that cannot be expressed in a predefined fashion. It stores the main characteristics of data by mapping it onto a numerical vector. An embedding is often unsupervised and constructed by transfer learning from large-scale unannotated data. Given an embedding, a downstream learning method, referred to as a two-stage method, is applicable to unstructured data. In this article, we introduce a novel framework of embedding learning to deliver a higher learning accuracy than the two-stage method while identifying an optimal learning-adaptive embedding. In particular, we propose a concept of U-minimal sufficient learning-adaptive embeddings, based on which we seek an optimal one to maximize the learning accuracy subject to an embedding constraint. Moreover, when specializing the general framework to classification, we derive a graph embedding classifier based on a hyperlink tensor representing multiple hypergraphs, directed or undirected, characterizing multi-way relations of unstructured data. Numerically, we design algorithms based on blockwise coordinate descent and projected gradient descent to implement linear and feed-forward neural network classifiers, respectively. Theoretically, we establish a learning theory to quantify the generalization error of the proposed method. Moreover, we show, in linear regression, that the one-hot encoder is more preferable among two-stage methods, yet its dimension restriction hinders its predictive performance. For a graph embedding classifier, the generalization error matches up to the standard fast rate or the parametric rate for linear or nonlinear classification. Finally, we demonstrate the utility of the classifiers on two benchmarks in grammatical classification and sentiment analysis.