A Segmentation Matrix Method for Chinese Segmentation Ambiguity Analysis
A Segmentation Matrix Method for Chinese Segmentation Ambiguity Analysis
复制标题
中文分词歧义分析的分词矩阵法
DOI：
--
复制
发表时间：
2016-06
期刊：
影响因子：
--
通讯作者：
Deil Zheng
中科院分区：
文献类型：
--
作者：
Yanping Chen;Qinghua Zheng;Feng Tian;Deil Zheng
Chinese Segmentation Ambiguity (CSA) is a fundamental problem confronted when processing Chinese language, where a sentence can generate more than one segmentation paths. Two techniques are commonly used to identify CSA: Omni-segmentation and Bi-directional Maximum Matching (BiMM). Due to the high computational complexity, Omni-segmentation is difficult to be applied for big data. BiMM is easier to be implemented and has a higher speed. However, recall of BiMM is much lower. In this paper, a Segmentation Matrix (SM) method is presented, which encodes each sentence as a matrix, then maps string operation into set operations. To identify CSA, instead of scanning a whole sentence, only specific areas of the matrix are checked. SM has a computational complexity close to BiMM with recall the same as Omni-segmentation. In addition to CSA identification, SM also supports lexicon-based Chinese word segmentation. In our experiments, based on SM, several issues about CSA are explored. The result shows that SM is useful for CSA analysis.
登录
查看更多内容
DOI：
--
发表时间：
2011-06
期刊：
--
影响因子：
--
作者：
Weiwei SUN
通讯作者：
Weiwei SUN
集成基于生成和判别字符的模型进行中文分词
DOI：
10.1145/2184436.2184440
发表时间：
2012-06
期刊：
ACM Transactions on Asian Language Information Processing
影响因子：
--
作者：
Kun Wang;Chengqing Zong;Keh-Yih Su
通讯作者：
Keh-Yih Su
影响因子：
6.4
作者：
Chen, Yanping;Zheng, Qinghua;Chen, Ping
通讯作者：
Chen, Ping
DOI：
10.1145/258525.258534
发表时间：
1997-07
期刊：
--
影响因子：
--
作者：
Lee-Feng Chien
通讯作者：
Lee-Feng Chien
DOI：
--
发表时间：
2008
期刊：
--
影响因子：
--
作者：
Wei Jin
通讯作者：
Wei Jin