GLAZE: Protecting Artists from Style Mimicry by Text-to-Image Models
GLAZE: Protecting Artists from Style Mimicry by Text-to-Image Models
复制标题
DOI：
10.48550/arxiv.2302.04222
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
Shawn Shan;Jenna Cryan;Emily Wenger;Haitao Zheng;Rana Hanocka;Ben Y. Zhao
中科院分区：
文献类型：
--
作者：
Shawn Shan;Jenna Cryan;Emily Wenger;Haitao Zheng;Rana Hanocka;Ben Y. Zhao
最近的文本到图像的扩散模型，如MidJourney和Stable diffusion，可能会取代专业艺术家群体中的许多人。特别是，模型可以在对特定艺术家的艺术样本进行“微调”后学习模仿他们的艺术风格。在本文中，我们描述了Glaze的设计、实现和评估，这是一个让艺术家在在线分享之前为他们的艺术应用“风格斗篷”的工具。这些斗篷对图像施加几乎无法察觉的扰动，当用作训练数据时，会误导试图模仿特定艺术家的生成模型。在与专业艺术家社区的协调下，我们对1000多名艺术家进行了用户研究，评估他们对人工智能艺术的看法，以及我们的工具的有效性，其可用性和对扰动的容忍度，以及在不同场景下和针对自适应对策的稳健性。被调查的艺术家和基于经验的基于clip的分数都表明，即使在低扰动水平（p=0.05）下，Glaze在正常条件下（>92%）和对适应性对策（>85%）的破坏模仿方面都非常成功。
Recent text-to-image diffusion models such as MidJourney and Stable Diffusion threaten to displace many in the professional artist community. In particular, models can learn to mimic the artistic style of specific artists after"fine-tuning"on samples of their art. In this paper, we describe the design, implementation and evaluation of Glaze, a tool that enables artists to apply"style cloaks"to their art before sharing online. These cloaks apply barely perceptible perturbations to images, and when used as training data, mislead generative models that try to mimic a specific artist. In coordination with the professional artist community, we deploy user studies to more than 1000 artists, assessing their views of AI art, as well as the efficacy of our tool, its usability and tolerability of perturbations, and robustness across different scenarios and against adaptive countermeasures. Both surveyed artists and empirical CLIP-based scores show that even at low perturbation levels (p=0.05), Glaze is highly successful at disrupting mimicry under normal conditions (>92%) and against adaptive countermeasures (>85%).