Basic research concerning adjacency probabilities in the development of a morphological analysis dictionary for classical Japanese poetry

首席调查员曾在2007年开发了一种对哇卡诗歌进行形态处理的工具。然而，它的适用范围仅限于八代树。本研究的目的是利用先前标注的切分数据和八代书的标记邻接概率，自动切分和标注《日本经济》的词性标签。使用KyTea(京都文本分析工具包)语素分割工具包，使用其默认的L2正则化支持向量机学习算法，模型学习只需不到一分钟。该模型在《日本经济新闻》上也取得了96%左右的高分割准确率。虽然关于未知词汇的添加和未知单词周围邻接概率的学习仍有一些剩余的工作要做，但可以认为已经完成了能够高精度切分《新时代》的词典的开发。

The principal investigator has previously developed a tool for the morphological processing of waka poems in 2007. However, its range of applicability was limited to the Hachidaishu. The goal of the present research is to automatically segment and annotate part-of-speech tags for the Nijuichidaishu using the previously annotated segmentation data and token adjacency probabilities of the Hachidaishu. Using the KyTea (Kyoto Text Analysis Toolkit) morpheme segmentation toolkit, with its default L2 regularized SVM learning algorithm, model learning took less than a minute. This model also achieved a high segmentation accuracy of around 96% on the Nijuichidaishu. While there is some remaining work to be done concerning the addition of unknown tokens and the learning of adjacency probabilities around unknown words, the development of a dictionary that can segment the Nijuichidaishu with a high accuracy can be considered complete.