论文标题
Gretel：图形对比主题增强语言模型的长文档提取性摘要
GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarization
论文作者
论文摘要
最近，神经主题模型（NTMS）已纳入预训练的语言模型（PLM）中，以捕获用于文本摘要的全局语义信息。但是，在这些方法中，它们捕获和整合全球语义信息的方式仍然存在局限性。在本文中，我们提出了一个新颖的模型，即图形对比主题增强语言模型（Gretel），该模型将图形对比主题模型与预训练的语言模型结合在一起，以充分利用长文档提取性摘要的全球和本地上下文语义。为了更好地捕获并将全局语义信息纳入PLM，图形对比主题模型集成了层次变压器编码器和图形对比度学习，以融合来自全局文档上下文和金摘要的语义信息。为此，Gretel鼓励该模型有效提取与黄金摘要有关的显着句子，而不是涵盖亚最佳主题的多余句子。对通用域和生物医学数据集的实验结果表明，我们所提出的方法的表现优于SOTA方法。
Recently, neural topic models (NTMs) have been incorporated into pre-trained language models (PLMs), to capture the global semantic information for text summarization. However, in these methods, there remain limitations in the way they capture and integrate the global semantic information. In this paper, we propose a novel model, the graph contrastive topic enhanced language model (GRETEL), that incorporates the graph contrastive topic model with the pre-trained language model, to fully leverage both the global and local contextual semantics for long document extractive summarization. To better capture and incorporate the global semantic information into PLMs, the graph contrastive topic model integrates the hierarchical transformer encoder and the graph contrastive learning to fuse the semantic information from the global document context and the gold summary. To this end, GRETEL encourages the model to efficiently extract salient sentences that are topically related to the gold summary, rather than redundant sentences that cover sub-optimal topics. Experimental results on both general domain and biomedical datasets demonstrate that our proposed method outperforms SOTA methods.
