论文标题
最接近风格可控生成的邻居语言模型
Nearest Neighbor Language Models for Stylistic Controllable Generation
论文作者
论文摘要
使用外部内存可以极大地提高语言建模性能。此内存编码上下文，以便在解码过程中可以召回类似的上下文。这种相似性取决于模型学习如何编码上下文，可以更改以包括其他属性，例如样式。为此，我们使用对礼貌，形式和毒性注释的语料库来构建和评估建筑。通过广泛的实验和人类评估，我们证明了我们在控制样式时产生文本的潜力。我们发现，特定于样式的数据座改善了生成性能，尽管结果各不相同，并且在将来的工作中应探讨数据和特定样式的效果。
Recent language modeling performance has been greatly improved by the use of external memory. This memory encodes the context so that similar contexts can be recalled during decoding. This similarity depends on how the model learns to encode context, which can be altered to include other attributes, such as style. We construct and evaluate an architecture for this purpose, using corpora annotated for politeness, formality, and toxicity. Through extensive experiments and human evaluation we demonstrate the potential of our method to generate text while controlling style. We find that style-specific datastores improve generation performance, though results vary greatly across styles, and the effect of pretraining data and specific styles should be explored in future work.
