论文标题
通过语义相似性学习使抽象性摘要更好
Learning by Semantic Similarity Makes Abstractive Summarization Better
论文作者
论文摘要
通过利用预训练的语言模型，摘要模型最近取得了迅速的进步。但是，这些模型主要通过自动评估指标（例如Rouge）评估。尽管Rouge以与人类评估分数有正相关而闻名，但它因其脆弱性和实际品质之间的差距而受到批评。在本文中，我们使用拥挤的人类评估度量标准比较了最近的LM，BART和基准数据集CNN/DM的参考摘要的生成摘要。有趣的是，相对于参考摘要，模型生成的摘要获得了更高的分数。源于我们的实验结果，我们首先论证了CNN/DM数据集的内在特征，预训练的语言模型的进度以及它们在训练数据上概括的能力。最后，我们分享了对模型生成的摘要的见解，并提出了我们对抽象性摘要的学习方法的思想。
By harnessing pre-trained language models, summarization models had rapid progress recently. However, the models are mainly assessed by automatic evaluation metrics such as ROUGE. Although ROUGE is known for having a positive correlation with human evaluation scores, it has been criticized for its vulnerability and the gap between actual qualities. In this paper, we compare the generated summaries from recent LM, BART, and the reference summaries from a benchmark dataset, CNN/DM, using a crowd-sourced human evaluation metric. Interestingly, model-generated summaries receive higher scores relative to reference summaries. Stemming from our experimental results, we first argue the intrinsic characteristics of the CNN/DM dataset, the progress of pre-trained language models, and their ability to generalize on the training data. Finally, we share our insights into the model-generated summaries and presents our thought on learning methods for abstractive summarization.
