论文标题
带有句子级目标的预训练变压器模型，用于答案句子选择
Pre-training Transformer Models with Sentence-Level Objectives for Answer Sentence Selection
论文作者
论文摘要
设计质量检查系统的一个重要任务是回答句子选择（AS2）：从一组检索到的相关文档中选择包含（或组成）问题的句子。在本文中，我们提出了三个新颖的句子级变压器预训练目标，这些目标纳入文档内和跨文档中的段落级语义，以提高变压器的AS2的性能，并减轻大型标记数据集的需求。具体而言，该模型的任务是预测：（i）从同一段落中提取两个句子，（ii）从给定段落中提取给定句子，（iii）从同一文档中提取了两个段落。我们对三个公共和一个工业AS2数据集进行的实验证明了我们预先训练的变压器比基线模型（例如Roberta和Electra）的经验优势。
An important task for designing QA systems is answer sentence selection (AS2): selecting the sentence containing (or constituting) the answer to a question from a set of retrieved relevant documents. In this paper, we propose three novel sentence-level transformer pre-training objectives that incorporate paragraph-level semantics within and across documents, to improve the performance of transformers for AS2, and mitigate the requirement of large labeled datasets. Specifically, the model is tasked to predict whether: (i) two sentences are extracted from the same paragraph, (ii) a given sentence is extracted from a given paragraph, and (iii) two paragraphs are extracted from the same document. Our experiments on three public and one industrial AS2 datasets demonstrate the empirical superiority of our pre-trained transformers over baseline models such as RoBERTa and ELECTRA for AS2.
