Linking Vision and Language through Computational Modelling

通过计算建模连接视觉和语言

• 批准号: 19K12733
• 负责人: CHANG Franklin
• 金额: $ 2.83万
• 依托单位: Kobe City University of Foreign Studies
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (C)
• 财政年份: 2019
• 资助国家: 日本
• 起止时间: 2019-04-01 至 2024-03-31
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-19K12733/
• 关键词: Vision; Language; Learning; Event understanding; Computational model; Deep Learning; Priming; Verbs; Syntax; Eyetracking; language; thematic roles; object tracking; connectionist model

I have written a paper on a deep learning model of how visual input is used to select verbs. The model is trained on data from human experiments on adults and the model was tested by comparing its verb use to children and adults. This paper has been accepted at Cognitive Science.

我写了一篇关于如何使用视觉输入来选择动词的深度学习模型的论文。该模型是根据成人的人体实验数据进行训练的，并通过比较儿童和成人的动词使用情况来测试该模型。这篇论文已被《认知科学》杂志接受。

项目成果

Abstract structures and meaning in Japanese dative structural priming

日语与格结构启动中的抽象结构和意义

• DOI: 10.1017/s0142716421000576
• 发表时间: 2022
• 期刊: Applied Psycholinguistics
• 影响因子: 2.1
• 作者: Chang Franklin;Tsumura Saki;Minemi Itsuki;Hirose Yuki
• 通讯作者: Hirose Yuki

Thematic role information is maintained in the visual object-tracking system

主题角色信息保存在视觉对象跟踪系统中

• DOI: 10.1177/1747021819882842
• 发表时间: 2019
• 期刊: Quarterly Journal of Experimental Psychology
• 影响因子: 1.7
• 作者: Jessop Andrew;Chang Franklin
• 通讯作者: Chang Franklin

Thematic role tracking difficulties across multiple visual events influences role use in language production

跨多个视觉事件的主题角色跟踪困难影响语言生成中的角色使用

• DOI: 10.1080/13506285.2021.2013374
• 发表时间: 2022
• 期刊: Visual Cognition
• 影响因子: 2
• 作者: Jessop Andrew;Chang Franklin
• 通讯作者: Chang Franklin

A deep learning verb production model using input from visual animations

使用视觉动画输入的深度学习动词生成模型

• 发表时间: 2022
• 作者: Chang;F.;Tatsumi;T.; Hiranuma;Y.;& Bannard;C.
• 通讯作者: C.