Variance of Average Surprisal: A Better Predictor for Quality of Grammar from Unsupervised PCFG Induction
Variance of Average Surprisal: A Better Predictor for Quality of Grammar from Unsupervised PCFG Induction
复制标题
DOI：
10.18653/v1/p19-1235
复制
发表时间：
2019-07
期刊：
影响因子：
--
通讯作者：
Lifeng Jin;William Schuler
中科院分区：
文献类型：
--
作者：
Lifeng Jin;William Schuler
在无监督语法归纳中，数据可能性与解析准确性只有弱相关性，特别是在多次运行后的收敛。为了找到一个更好的指标，诱导语法的质量，本文相关的几个语言和心理语言动机的预测分析准确性的大型多语种语法归纳评估数据集。结果表明，方差的平均似然（VAS）更好地与解析精度比数据的可能性和使用VAS，而不是数据的可能性模型选择提供了显着的准确性提高。进一步的证据表明，VAS是一个更好的候选人比数据的可能性预测词序类型分类。分析表明，VAS似乎将自然语言语法中的实词与虚词分开，并且更好地将具有不同频率的词安排到更符合语言学理论的单独类中。
In unsupervised grammar induction, data likelihood is known to be only weakly correlated with parsing accuracy, especially at convergence after multiple runs. In order to find a better indicator for quality of induced grammars, this paper correlates several linguistically- and psycholinguistically-motivated predictors to parsing accuracy on a large multilingual grammar induction evaluation data set. Results show that variance of average surprisal (VAS) better correlates with parsing accuracy than data likelihood and that using VAS instead of data likelihood for model selection provides a significant accuracy boost. Further evidence shows VAS to be a better candidate than data likelihood for predicting word order typology classification. Analyses show that VAS seems to separate content words from function words in natural language grammars, and to better arrange words with different frequencies into separate classes that are more consistent with linguistic theory.