Vision and language cross-modal for training conditional GANs with long-tail data.

我们学习视觉和语言空间之间的交叉模态。本文的主要研究成果有三：1.从开放词典Wiktionary中收集了一组对象的名称和定义，并使用预先训练好的BERT模型嵌入定义。我们将这种外部知识的图像字幕模型，优于其他方法在新的对象字幕任务。它发表在CVPR 2022.2上。我们通过使用翻转和非翻转非饱和损耗，提出了一种新的GAN训练方案。3.我们创建了一个新的故事评估数据集，包括通过Reddit网站和众包注释过程收集的10万个故事排名数据和46k方面评级和推理。它发表在EMNLP 2022上。

We learn the cross-modality between vision and language spaces. We obtained three achievements:1.We collected a set of objects' names and definitions from the open dictionary Wiktionary and used the pre-trained BERT model to embed the definitions. We incorporated this external knowledge into an image captioning model, outperforming other methods in novel object captioning task. It was published at CVPR 2022.2.We proposed a new training scheme for GANs by using flipped and non-flipped non-saturating losses. It was published in the IEEE Access journal (IF 3.476).3.We created a new dataset for story evaluation, consisting of 100k story ranking data and 46k aspect rating and reasoning collected through the Reddit website and crowd-sourcing annotation process. It was published at EMNLP 2022.