Leveraging label hierarchy using transfer and multi-task learning: A case study on patent classification
Leveraging label hierarchy using transfer and multi-task learning: A case study on patent classification
复制标题
DOI：
10.1016/j.neucom.2021.07.057
复制
发表时间：
2021-07
期刊：
影响因子：
6
通讯作者：
S. Aroyehun;Jason Angel;Navonil Majumder;Alexander Gelbukh;A. Hussain
中科院分区：
文献类型：
--
作者：
S. Aroyehun;Jason Angel;Navonil Majumder;Alexander Gelbukh;A. Hussain
当标签被组织成有意义的分类时，不同级别的标签之间的父子关系可以为分类器提供仅从数据中无法推断的额外信息，特别是在训练数据有限的情况下。作为一个案例研究，我们说明了这对专利分类任务的影响--根据专利文件的技术含量对其进行分类的任务。现有的方法没有考虑到这一额外信息。在两个专利分类数据集WIPO-Alpha和USPTO-2M上的实验表明，我们的正则化门控递归单元(GRU)结构已经在这两个数据集上分别使用0.5191和0.5740的最高预测提高了性能，获得了微平均精度分数。然而，沿着标签层次进行的知识转移使WIPO-Alpha的得分进一步显著提高，得分提高到0.5376，而USPTO-2M的得分略有提高，达到0.5743。我们的分析表明，纳入标签信息提高了在样本较少的类别上的性能，并使模型对预测密切相关的标签所产生的错误具有健壮性。
When labels are organized into a meaningful taxonomy, the parent-child relationship between labels at different levels can give the classifier additional information not deducible from the data alone, especially with limited training data. As a case study, we illustrate this effect on the task of patent classification—the task of categorizing patent documents based on their technical content. Existing approaches do not take into consideration this additional information. Experiments on two patent classification datasets, WIPO-alpha and USPTO-2M, show that our regularized Gated Recurrent Unit (GRU) architecture already gives a performance improvement with a micro-averaged precision score using the top prediction of 0.5191 and 0.5740 on the two datasets, respectively. However, knowledge transfer along the label hierarchy gives further significant improvement on WIPO-alpha, raising the score to 0.5376, and a small improvement on USPTO-2M to 0.5743. Our analyses reveal that incorporating label information improves performance on classes with fewer examples and makes model robust to errors that result from predicting closely related labels.