论文标题
冷解码：基于能量的限制性文本生成Langevin Dynamics
COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics
论文作者
论文摘要
文本生成的许多应用都需要合并不同的约束来控制生成的文本的语义或样式。这些约束可能很困难（例如，确保输出中包含某些关键字）和软（例如，用左侧或右侧的上下文将输出对输出进行上下文化）。在本文中，我们提出了基于能量的约束解码（Cold），这是一个解码框架，将约束的生成统一为通过能量函数指定约束，然后通过基于梯度的采样来对约束进行有效的可区分推理。冷解码是一个灵活的框架，可以直接应用于从左到右的语言模型，而无需任何特定于任务的微调，如三个具有挑战性的文本生成应用所证明：词汇约束的生成，绑架推理和反事实推理。我们对这些受约束生成任务的实验表明，无论是自动和人类评估而言，我们的方法的有效性。
Many applications of text generation require incorporating different constraints to control the semantics or style of generated text. These constraints can be hard (e.g., ensuring certain keywords are included in the output) and soft (e.g., contextualizing the output with the left- or right-hand context). In this paper, we present Energy-based Constrained Decoding with Langevin Dynamics (COLD), a decoding framework which unifies constrained generation as specifying constraints through an energy function, then performing efficient differentiable reasoning over the constraints through gradient-based sampling. COLD decoding is a flexible framework that can be applied directly to off-the-shelf left-to-right language models without the need for any task-specific fine-tuning, as demonstrated through three challenging text generation applications: lexically-constrained generation, abductive reasoning, and counterfactual reasoning. Our experiments on these constrained generation tasks point to the effectiveness of our approach, both in terms of automatic and human evaluation.
