LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation
LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation
复制标题
DOI：
10.48550/arxiv.2305.11116
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
Yujie Lu;Xianjun Yang;Xiujun Li;X. Wang;William Yang Wang
中科院分区：
文献类型：
--
作者：
Yujie Lu;Xianjun Yang;Xiujun Li;X. Wang;William Yang Wang
现有的文本到图像合成的自动评价只能提供一个图像-文本匹配的分数，没有考虑对象级的组合性，这导致与人类判断的相关性差。在这项工作中，我们提出了LLMScore，一个新的框架，提供多粒度组合的评价分数。LLMScore利用大型语言模型（LLM）来评估文本到图像模型。最初，它将图像转换为图像级和对象级的视觉描述。然后，将评估指令馈送到LLM中以测量合成图像和文本之间的对齐，最终生成带有理由的分数。我们的大量分析揭示了LLMScore与人类对各种数据集（属性绑定对比，概念连接，MSCOCO，DrawBench，PaintSkills）的判断的最高相关性。值得注意的是，我们的LLMScore实现了与人类评估的Kendall tau相关性，分别比常用的文本图像匹配度量CLIP和BLIP高出58.8%和31.2%。
Existing automatic evaluation on text-to-image synthesis can only provide an image-text matching score, without considering the object-level compositionality, which results in poor correlation with human judgments. In this work, we propose LLMScore, a new framework that offers evaluation scores with multi-granularity compositionality. LLMScore leverages the large language models (LLMs) to evaluate text-to-image models. Initially, it transforms the image into image-level and object-level visual descriptions. Then an evaluation instruction is fed into the LLMs to measure the alignment between the synthesized image and the text, ultimately generating a score accompanied by a rationale. Our substantial analysis reveals the highest correlation of LLMScore with human judgments on a wide range of datasets (Attribute Binding Contrast, Concept Conjunction, MSCOCO, DrawBench, PaintSkills). Notably, our LLMScore achieves Kendall's tau correlation with human evaluations that is 58.8% and 31.2% higher than the commonly-used text-image matching metrics CLIP and BLIP, respectively.