Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation
Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation
复制标题
DOI：
10.1109/iccv48922.2021.00537
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
Zechen Bai;Yuta Nakashima;Noa García
中科院分区：
文献类型：
--
作者：
Zechen Bai;Yuta Nakashima;Noa García
你有没有看过一幅画，想知道它背后的故事是什么？这项工作提出了一个框架，使艺术更接近人产生全面的描述美术画。然而，为艺术品生成信息描述是非常具有挑战性的，因为它需要1）描述图像的多个方面，例如其风格，内容或组成，以及2）提供有关艺术家，其影响或历史时期的背景和上下文知识。为了解决这些挑战，我们引入了一个多主题和知识渊博的艺术描述框架，该框架根据三个艺术主题生成的句子模块，此外，还利用外部知识增强了每个描述。该框架是通过详尽的分析，定量和定性，以及比较人的评价验证，展示了突出的成果，在主题的多样性和信息的准确性。
Have you ever looked at a painting and wondered what is the story behind it? This work presents a framework to bring art closer to people by generating comprehensive descriptions of fine-art paintings. Generating informative descriptions for artworks, however, is extremely challenging, as it requires to 1) describe multiple aspects of the image such as its style, content, or composition, and 2) provide background and contextual knowledge about the artist, their influences, or the historical period. To address these challenges, we introduce a multi-topic and knowledgeable art description framework, which modules the generated sentences according to three artistic topics and, additionally, enhances each description with external knowledge. The framework is validated through an exhaustive analysis, both quantitative and qualitative, as well as a comparative human evaluation, demonstrating outstanding results in terms of both topic diversity and information veracity.