论文标题
文本生成的统一复杂性
Uniform Complexity for Text Generation
论文作者
论文摘要
大型语言模型（LLMS）在各种生成的NLP任务（例如摘要和机器翻译）中显示出令人鼓舞的结果。但是，在叙事生成的背景下，现有模型仍然没有捕获有助于产生一致文本的因素。例如，合乎逻辑的是，整个文本或故事应该在整个过程中都可以统一阅读，并且这种复杂性应该可以控制。因此，如果输入文本提示的复杂性在Flesch Reading Ease测试中被评为一年级阅读级别，则生成的文本继续绘图也应在此复杂性范围内。考虑到这一点，我们引入了文本生成（UCTG）统一的复杂性，这是一种新的基准测试，这引发了使生成模型观察到提示方面统一的语言特性的挑战。我们在语言和认知上有150多种以上的特征，以评估人类和生成模型中的文本复杂性。从我们的结果来看，我们发现，诸如GPT-2之类的模型努力保持世代相传的输入提示的复杂性，即使对专业书面文本进行了填补。
Large language models (LLMs) have shown promising results in a wide array of generative NLP tasks, such as summarization and machine translation. In the context of narrative generation, however, existing models still do not capture factors that contribute to producing consistent text. For instance, it is logical that a piece of text or a story should be uniformly readable throughout and that this form of complexity should be controllable. As such, if the complexity of an input text prompt is rated first-grade reading level in the Flesch Reading Ease test, then the generated text continuing the plot should also be within this range of complexity. With this in mind, we introduce Uniform Complexity for Text Generation (UCTG), a new benchmark test which raises the challenge of making generative models observe uniform linguistic properties with respect to prompts. We experiment with over 150+ linguistically and cognitively motivated features for evaluating text complexity in humans and generative models. From our results, we find that models such as GPT-2 struggle to preserve the complexity of input prompts used in its generations, even if finetuned with professionally written texts.
