Recognizing Phrasal Entailments using Image Groundings

使用图像基础识别短语蕴涵

• 批准号: 17K12747
• 负责人: 韓 丹
• 金额: $ 2.66万
• 依托单位: National Institute of Advanced Industrial Science and Technology
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Young Scientists (B)
• 财政年份: 2017
• 资助国家: 日本
• 起止时间: 2017-04-01 至 2021-03-31
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-17K12747/
• 关键词: Image groundings; visual denotations; textual entailment; multimodality; Image grounding; RTE; Lexical semantics; NLP

For this project, I created an effective technique to efficiently extract phrases inspired by bilingual extractions of tree structures. I also designed and evaluated a method to compare sets of images and assess their entailment relation which was also new in the field. Finally, I integrated these innovations in an end-to-end system to measure their impact in the task of Recognizing Textual Entailment, observing a net increase in accuray of 1.2%. My strategy was most successful with nouns, noun phrases and adjectives + nouns; however, it was not effective for most verb phrases and abstract nouns since these phrases cannot be well grounded into images. This worked has been published in JSAI 2017 and EMNLP 2017 which has received 9 citations from other impactful conferences and journals.

对于这个项目，我创建了一个有效的技术来有效地提取短语，灵感来自于树结构的双语提取。我还设计并评估了一种方法来比较图像集并评估它们的蕴涵关系，这在该领域也是新的。最后，我将这些创新集成到一个端到端系统中，以衡量它们在识别文本蕴涵任务中的影响，观察到准确率净增加1.2%。我的策略在名词、名词短语和形容词+名词上最成功;然而，它对大多数动词短语和抽象名词无效，因为这些短语不能很好地扎根于图像。这项工作已发表在JSAI 2017和EMNLP 2017上，已收到其他有影响力的会议和期刊的9次引用。

项目成果

Visual Denotations for Recognizing Textual Entailment

• DOI: 10.18653/v1/d17-1305
• 发表时间: 2017-09
• 作者: D. Han;Pascual Martínez-Gómez;K. Mineshima

Grounded noun/verb-phrases to images for RTE

RTE 图像中的接地名词/动词短语

• 发表时间: 2017
• 期刊: In Proc. of the 31st Annual Conference of the Japanese Society for Artificial Intelligence
• 作者: Dan Han; Pascual Martinez-Gomez
• 通讯作者: Pascual Martinez-Gomez