论文标题
最近的邻居非自动回归文本生成
Nearest Neighbor Non-autoregressive Text Generation
论文作者
论文摘要
非自动回旋（NAR）模型的计算能力比自回归模型少，但牺牲生成质量就可以生成较少的计算句子。先前的研究通过迭代解码解决了这个问题。这项研究建议将最近的邻居用作NAR解码器的初始状态，并迭代编辑。我们提出了一种新颖的培训策略，以了解有关邻居的编辑操作，以改善NAR文本生成。实验结果表明，所提出的方法（邻域）在JRC-ACQUISIE EN-DE DATASET上的解码迭代率更少（少于1.69点比香草变压器高1.69点），该迭代率较少（少于十次的迭代），即使用邻居的机器转换的常见基准标准数据集，用于使用邻居。我们还确认了所提出的方法对数据到文本任务（Wikibio）的有效性。另外，所提出的方法在WMT'14 EN-DE数据集上优于NAR基线。我们还报告了建议方法中使用的邻居实例的分析。
Non-autoregressive (NAR) models can generate sentences with less computation than autoregressive models but sacrifice generation quality. Previous studies addressed this issue through iterative decoding. This study proposes using nearest neighbors as the initial state of an NAR decoder and editing them iteratively. We present a novel training strategy to learn the edit operations on neighbors to improve NAR text generation. Experimental results show that the proposed method (NeighborEdit) achieves higher translation quality (1.69 points higher than the vanilla Transformer) with fewer decoding iterations (one-eighteenth fewer iterations) on the JRC-Acquis En-De dataset, the common benchmark dataset for machine translation using nearest neighbors. We also confirm the effectiveness of the proposed method on a data-to-text task (WikiBio). In addition, the proposed method outperforms an NAR baseline on the WMT'14 En-De dataset. We also report analysis on neighbor examples used in the proposed method.
