论文标题
苏格拉底预审计：可控摘要的问题驱动的预段
Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization
论文作者
论文摘要
在可控数据的长期可控摘要中，标记的数据稀缺，经过审计的模型难以适应任务并有效响应用户查询。在本文中，我们介绍了苏格拉底式预审查，这是一个问题驱动的，无监督的预定目标，专门设计用于提高摘要任务中的可控性。通过训练模型在给定情况下生成和回答相关问题，苏格拉底式预告片使该模型能够更有效地遵守用户提供的查询并确定要汇总的相关内容。我们通过对两个摘要领域，短篇小说和对话以及多种控制策略进行广泛的实验来证明这种方法的有效性：关键字，问题和FACTOID QA对。我们的训练方法仅依赖于未标记的文档和问题生成系统，并且胜过使用其他监督数据的预先调查方法。此外，我们的结果表明，苏格拉底式预训练削减了一半的特定任务标签数据要求，更忠实于用户提供的查询，并在QMSUM和Squality上实现最先进的性能。
In long document controllable summarization, where labeled data is scarce, pretrained models struggle to adapt to the task and effectively respond to user queries. In this paper, we introduce Socratic pretraining, a question-driven, unsupervised pretraining objective specifically designed to improve controllability in summarization tasks. By training a model to generate and answer relevant questions in a given context, Socratic pretraining enables the model to more effectively adhere to user-provided queries and identify relevant content to be summarized. We demonstrate the effectiveness of this approach through extensive experimentation on two summarization domains, short stories and dialogue, and multiple control strategies: keywords, questions, and factoid QA pairs. Our pretraining method relies only on unlabeled documents and a question generation system and outperforms pre-finetuning approaches that use additional supervised data. Furthermore, our results show that Socratic pretraining cuts task-specific labeled data requirements in half, is more faithful to user-provided queries, and achieves state-of-the-art performance on QMSum and SQuALITY.
