NEUROSTRUCTURAL DECODING: Neural Text Generation with Structural Constraints
NEUROSTRUCTURAL DECODING: Neural Text Generation with Structural Constraints
复制标题
DOI：
10.18653/v1/2023.acl-long.528
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Mohaddeseh Bastan;M. Surdeanu;Niranjan Balasubramanian
中科院分区：
文献类型：
--
作者：
Mohaddeseh Bastan;M. Surdeanu;Niranjan Balasubramanian
文本生成通常涉及生成连贯且语法正确的文本，这些文本也满足一组给定的语义约束。虽然大多数条件文本生成方法主要关注词汇约束，但它们往往难以有效地合并句法约束，这为近似语义约束提供了更丰富的语言。我们通过引入 NeuroStructural Decoding 来解决这一差距，这是一种新的解码算法，它合并了句法约束，以进一步提高生成文本的质量。我们在 NeuroLogic Decoding (Lu etal. 2021) 算法的基础上构建了 NeuroStructural Decoding，该算法使语言生成模型能够生成流畅的文本，同时满足复杂的词汇约束。我们的算法功能强大且可扩展。它在解码过程中通过解析每一步的部分生成来跟踪词汇句法约束（例如，我们需要观察狗作为主语，球作为宾语）。为此，我们采用依存解析器来生成不完整句子的解析。我们的方法在三种不同的语言生成任务上进行了评估，结果表明与以前的方法相比，词汇和句法指标的性能都有所提高。结果表明，这是将细粒度可控生成集成到传统波束搜索解码中的有前途的解决方案。
Text generation often involves producing coherent and grammatically correct texts that also satisfy a given set of semantic constraints. While most approaches for conditional text generation have primarily focused on lexical constraints, they often struggle to effectively incorporate syntactic constraints, which provide a richer language for approximating semantic constraints.We address this gap by introducing NeuroStructural Decoding, a new decoding algorithm that incorporates syntactic constraints to further improve the quality of the generated text. We build NeuroStructural Decoding on the NeuroLogic Decoding (Lu etal. 2021) algorithm, which enables language generation models to produce fluent text while satisfying complex lexical constraints. Our algorithm is powerful and scalable. It tracks lexico-syntactic constraints (e.g., we need to observe dog as subject and ball as object)during decoding by parsing the partial generations at each step. To this end, we adapt a dependency parser to generate parses for incomplete sentences. Our approach is evaluated on three different language generation tasks, and the results show improved performance in both lexical and syntactic metrics compared to previous methods. The results suggest this is a promising solution for integrating fine-grained controllable generation into the conventional beam search decoding.