Unsupervised Learning of PCFGs with Normalizing Flow
Unsupervised Learning of PCFGs with Normalizing Flow
复制标题
DOI：
10.18653/v1/p19-1234
复制
发表时间：
2019-07
期刊：
影响因子：
--
通讯作者：
Lifeng Jin;F. Doshi-Velez;Timothy Miller;Lane Schwartz;William Schuler
中科院分区：
文献类型：
--
作者：
Lifeng Jin;F. Doshi-Velez;Timothy Miller;Lane Schwartz;William Schuler
无监督PCFG诱导假设集紧凑的上下文无关的规则解释的句子。PCFG归纳不仅为低资源语言提供了工具，而且在建模语言习得中发挥了重要作用（Bannard et al.，2009; Abend et al. 2017）。然而，目前的PCFG归纳模型，使用单词标记作为输入，是无法将语义和形态到归纳，并可能遇到的问题时，面对形态丰富的语言稀疏的词汇。本文描述了一种采用上下文嵌入的神经PCFG诱导器（Peters等人，2018）在归一化流模型（Dinh等人，2015）扩展PCFG归纳以使用语义和形态信息。语言动机的稀疏性和分类距离的限制施加在诱导正则化。实验表明，规范化流程的PCFG归纳模型在各种不同的语言上产生具有最先进精度的语法。消融进一步显示了规范化流，上下文嵌入和提出的正则化的积极影响。
Unsupervised PCFG inducers hypothesize sets of compact context-free rules as explanations for sentences. PCFG induction not only provides tools for low-resource languages, but also plays an important role in modeling language acquisition (Bannard et al., 2009; Abend et al. 2017). However, current PCFG induction models, using word tokens as input, are unable to incorporate semantics and morphology into induction, and may encounter issues of sparse vocabulary when facing morphologically rich languages. This paper describes a neural PCFG inducer which employs context embeddings (Peters et al., 2018) in a normalizing flow model (Dinh et al., 2015) to extend PCFG induction to use semantic and morphological information. Linguistically motivated sparsity and categorical distance constraints are imposed on the inducer as regularization. Experiments show that the PCFG induction model with normalizing flow produces grammars with state-of-the-art accuracy on a variety of different languages. Ablation further shows a positive effect of normalizing flow, context embeddings and proposed regularizers.