Towards easier and faster sequence labeling for natural language processing: A search-based probabilistic online learning framework (SAPO)
Towards easier and faster sequence labeling for natural language processing: A search-based probabilistic online learning framework (SAPO)
复制标题
为自然语言处理实现更简单、更快速的序列标记：基于搜索的概率在线学习框架（SAPO）
DOI：
10.1016/j.ins.2018.11.025
复制
发表时间：
2019-04-01
影响因子：
8.1
通讯作者：
Ren, Xuancheng
中科院分区：
文献类型：
--
作者：
Sun, Xu;Ma, Shuming;Ren, Xuancheng
有两种主要的序列标记方法。一种是基于概率梯度的方法，如条件随机场（CRF）和神经网络（如RNN），它们具有很高的准确性，但缺点是：训练缓慢，不支持基于搜索的优化（这在许多情况下很重要）。另一种是基于搜索的学习方法，如结构化感知器和边缘注入放松算法（MIRA），它们训练速度快，但也存在准确率低、没有概率信息、在现实任务中不收敛等缺点。我们提出了一种新颖而“简单”的解决方案，一种基于搜索的概率在线学习方法，来解决大多数这些问题。该方法“简单”，因为训练阶段的优化算法与测试阶段的解码算法一样简单。该方法搜索输出的候选对象，导出概率，并进行有效的在线学习。结果表明，该方法训练速度快，收敛性有理论保证，易于实现，能够支持基于搜索的优化并获得最高精度。在已知任务上的实验表明，该方法比CRF和BiLSTM具有更好的准确率。(1) (C) 2018 Elsevier Inc.版权所有。
There are two major approaches for sequence labeling. One is the probabilistic gradient-based methods such as conditional random fields (CRF) and neural networks (e.g., RNN), which have high accuracy but drawbacks: slow training, and no support of search-based optimization (which is important in many cases). The other is the search-based learning methods such as structured perceptron and margin infused relaxed algorithm (MIRA), which have fast training but also drawbacks: low accuracy, no probabilistic information, and non-convergence in real-world tasks. We propose a novel and "easy" solution, a search-based probabilistic online learning method, to address most of those issues. The method is "easy", because the optimization algorithm at the training stage is as simple as the decoding algorithm at the test stage. This method searches the output candidates, derives probabilities, and conducts efficient online learning. We show that this method with fast training and theoretical guarantee of convergence, which is easy to implement, can support search-based optimization and obtain top accuracy. Experiments on well-known tasks show that our method has better accuracy than CRF and BiLSTM.(1) (C) 2018 Elsevier Inc. All rights reserved.