Hierarchical Lifelong Learning by Sharing Representations and Integrating Hypothesis
Hierarchical Lifelong Learning by Sharing Representations and Integrating Hypothesis
复制标题
通过共享表征和整合假设进行分层终身学习
DOI：
10.1109/tsmc.2018.2884996
复制
发表时间：
--
期刊：
影响因子：
--
通讯作者：
Xiaofen Xing
中科院分区：
文献类型：
--
作者：
Tong Zhang;Guoxi Su;Chunmei Qing;Xiangmin Xu;Bolun Cai;Xiaofen Xing
在终身机器学习（LML）系统中，来自不断变化的环境的连续新任务被学习并添加到系统中。然而，在经典的监督式LML系统中，在转移知识之前，充分标记的数据对于提取任务间关系是必不可少的。由于初始近似较差，不充分的标签可能会降低性能。为了对典型的LML系统进行扩展，提出了一种新的分层终身学习算法（HLLA），该算法由以下两层组成：1)底层是由共享表示和集成知识库组成的知识层；2)顶层是由带有特征的参数化假设函数组成的知识层。HLLA利用未标记的数据对共享表示进行预训练。我们还考虑了一种选择性继承更新方法来处理任务间分布的变化。实验表明，我们的HLLA方法优于许多其他最近的LML算法，特别是在处理高维、低相关性和较少标记数据问题时。
In lifelong machine learning (LML) systems, consecutive new tasks from changing circumstances are learned and added to the system. However, sufficiently labeled data are indispensable for extracting intertask relationships before transferring knowledge in classical supervised LML systems. Inadequate labels may deteriorate the performance due to the poor initial approximation. In order to extend the typical LML system, we propose a novel hierarchical lifelong learning algorithm (HLLA) consisting of two following layers: 1) the knowledge layer consisted of shared representations and integrated knowledge basis at the bottom and 2) parameterized hypothesis functions with features at the top. Unlabeled data is leveraged in HLLA for pretraining of the shared representations. We also have considered a selective inherited updating method to deal with intertask distribution shifting. Experiments show that our HLLA method outperforms many other recent LML algorithms, especially when dealing with higher dimensional, lower correlation, and fewer labeled data problems.