Gradient-based Constrained Sampling from Language Models
Gradient-based Constrained Sampling from Language Models
复制标题
DOI：
10.18653/v1/2022.emnlp-main.144
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Sachin Kumar;Biswajit Paria;Yulia Tsvetkov
中科院分区：
文献类型：
--
作者：
Sachin Kumar;Biswajit Paria;Yulia Tsvetkov
大型的预训练语言模型在生成流畅的文本方面是成功的，但众所周知，很难可控地从中采样。在这项工作中，我们研究了这种语言模型的约束采样，即，生成满足用户定义的约束的文本，同时保持流畅性和模型在下游任务中的性能。我们提出了MuCoLa-一个采样过程，它结合了语言模型的对数似然性与任意（可微）的约束在一个单一的能量函数，然后生成样本的非自回归方式。具体地说，它将整个输出序列与噪声叠加，并使用该能量的梯度遵循由朗之万动力学定义的马尔可夫链。我们评估了MuCoLa在文本生成方面的软约束和硬约束以及它们的组合，在毒性避免，情感控制和关键词引导生成方面获得了显着的改进。
Large pretrained language models are successful at generating fluent text but are notoriously hard to controllably sample from. In this work, we study constrained sampling from such language models, i.e., generating text that satisfies user-defined constraints, while maintaining fluency and model’s performance in a downstream task. We propose MuCoLa—a sampling procedure that combines the log-likelihood of the language model with arbitrary (differentiable) constraints in a single energy function, and then generates samples in a non-autoregressive manner. Specifically, it initializes the entire output sequence with noise and follows a Markov chain defined by Langevin Dynamics using the gradients of this energy. We evaluate MuCoLa on text generation with soft and hard constraints as well as their combinations, obtaining significant improvements over competitive baselines for toxicity avoidance, sentiment control, and keyword-guided generation.