CAREER: Controllable generation for instruction-following language models

像ChatGPT这样的指令遵循语言模型开始得到广泛的发展，理解和控制这些系统的能力对于确保它们造福社会至关重要。尽管这些语言模型在生成流畅且令人信服的输出方面取得了成功，但越来越多的工作表明，这些系统可能会生成用户、模型创建者甚至整个社会不希望看到的输出。创建模仿人类的模型的能力与无法让它们满足特定需求（例如，拒绝生成不正确的信息）之间的差距表明，在精确控制这些系统的能力方面存在重大缺陷。这个项目旨在建立有原则的、透明的和精确的方法来控制语言模型。为了实现这些目标，本项目将可控生成视为一种可行的长期路径，以创建精确遵循我们设计目标的指令遵循语言模型。可控发电有几个好处。首先，它定义了一个精确的统计建模问题，在这个问题上可以建立有原则的方法和严格的评估。其次，它将控制目标从任务中分离出来，通过允许用户准确地看到模型设计者正在优化的内容来提高透明度。第三，它通过推理时间方法（如拒绝抽样）实现更精确的控制，这严格地将控制作为约束。虽然可控生成对语言模型具有主要的长期好处，但仍然存在必须首先解决的重大开放问题，包括执行离散搜索的困难，对专门训练的需求，以及缺乏现实的野外控制任务基准。我们将通过结合新模型（如基于扩散的模型）、零射击和基于解码器的控制方法以及野外控制行为的广泛基准来解决这些挑战。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Instruction-following language models like ChatGPT are beginning to see widespread development, and the ability to understand these systems and control them is critically important to make sure that they benefit society. Despite the success of these language models in generating fluent and convincing-looking outputs, there has been a growing body of work indicating that these systems can generate outputs that are undesirable to users, model creators, and even society at large. This gap between the ability to create models that imitate humans and the inability to have them fulfill specific desiderata (e.g. refuse to generate incorrect information) shows a major deficiency in the ability to precisely control these systems. This project aims to build principled, transparent, and precise methods for controlling language models.To achieve these goals, this project views controllable generation as a viable long-term path to creating instruction-following language models that precisely follow our design goals. Controllable generation offers several benefits. First, it defines a precise statistical modeling problem on which it is possible to build principled methods and rigorous evaluations. Second, it separates the control target from the task, improving transparency by allowing users to see exactly what is being optimized by the model designers. Third, it enables much more precise controls via inference-time methods such as rejection sampling, which strictly enforces the control as a constraint. While controllable generation has major long-term benefits for language models, there also remain significant open problems that must be resolved first, including the difficulty of performing discrete search, the need for specialized training, and the lack of realistic benchmarks of control tasks in the wild. We will address these challenges through a combination of new models (such as diffusion-based models), zero-shot and decoder-based control methods, and a broad benchmark of in-the-wild control behaviors.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.