Grounded PCFG Induction with Images
Grounded PCFG Induction with Images
复制标题
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Lifeng Jin;William Schuler
中科院分区：
文献类型：
--
作者：
Lifeng Jin;William Schuler
最近在无监督解析方面的工作试图将视觉信息纳入学习，但结果表明，这些模型需要语言偏见来与仅依赖文本的模型竞争。这项工作提出了语法归纳模型，该模型使用图像中的视觉信息进行标记分析，并在几种语言的基础语法归纳上取得了最先进的结果。结果表明，视觉信息在高频词分布广泛的语言中特别有用。有视觉信息和无视觉信息模型的比较表明，基于视觉信息的模型能够利用视觉信息提出名词短语，从图像中收集未知单词的有用信息，并且在介词短语依恋预测方面取得了更好的效果。
Recent work in unsupervised parsing has tried to incorporate visual information into learning, but results suggest that these models need linguistic bias to compete against models that only rely on text. This work proposes grammar induction models which use visual information from images for labeled parsing, and achieve state-of-the-art results on grounded grammar induction on several languages. Results indicate that visual information is especially helpful in languages where high frequency words are more broadly distributed. Comparison between models with and without visual information shows that the grounded models are able to use visual information for proposing noun phrases, gathering useful information from images for unknown words, and achieving better performance at prepositional phrase attachment prediction.