论文标题
标签语义知识的预训练，用于几个播种文本分类
Label Semantic Aware Pre-training for Few-shot Text Classification
论文作者
论文摘要
在文本分类任务中，有用的信息在标签名称中编码。标签语义知识系统已利用此信息在微调和预测过程中改善文本分类性能。但是，尚未广泛探索预训练期间的标签 - 仪器。因此，我们建议标签语义意识到的预训练（LSAP），以提高文本分类系统的概括和数据效率。 LSAP通过对来自各个域的标记句子进行次级预训练，将标签语义纳入预训练的生成模型（T5）中。由于域将军预训练需要大量数据，因此我们开发了一个过滤和标记管道，以自动从未标记的文本创建句子标签对。我们对意图（ATI，SNIPS，TOPV2）和主题分类（AG News，Yahoo！答案）进行实验。 LSAP比最先进的模型获得了明显的准确性改进，以进行几次弹头文本分类，同时保持与高资源环境中最新技术相当的性能。
In text classification tasks, useful information is encoded in the label names. Label semantic aware systems have leveraged this information for improved text classification performance during fine-tuning and prediction. However, use of label-semantics during pre-training has not been extensively explored. We therefore propose Label Semantic Aware Pre-training (LSAP) to improve the generalization and data efficiency of text classification systems. LSAP incorporates label semantics into pre-trained generative models (T5 in our case) by performing secondary pre-training on labeled sentences from a variety of domains. As domain-general pre-training requires large amounts of data, we develop a filtering and labeling pipeline to automatically create sentence-label pairs from unlabeled text. We perform experiments on intent (ATIS, Snips, TOPv2) and topic classification (AG News, Yahoo! Answers). LSAP obtains significant accuracy improvements over state-of-the-art models for few-shot text classification while maintaining performance comparable to state of the art in high-resource settings.
