论文标题
部分可观测时空混沌系统的无模型预测
TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models
论文作者
论文摘要
预先训练的语言模型已在自然语言处理中占了上风，并成为许多NLP任务的骨干，但是对计算资源的需求限制了其应用程序。在本文中，我们介绍了TextPruner，这是一种为预训练的语言模型而设计的开源模型修剪工具包，以快速轻松的模型压缩为目标。 TextPruner提供结构化的训练后修剪方法，包括词汇修剪和变压器修剪，可以应用于各种模型和任务。我们还提出了一种自我监督的修剪方法，无需标记数据就可以应用。我们使用多个NLP任务进行的实验证明了文本通用器在不重新训练模型的情况下降低模型大小的能力。
Pre-trained language models have been prevailed in natural language processing and become the backbones of many NLP tasks, but the demands for computational resources have limited their applications. In this paper, we introduce TextPruner, an open-source model pruning toolkit designed for pre-trained language models, targeting fast and easy model compression. TextPruner offers structured post-training pruning methods, including vocabulary pruning and transformer pruning, and can be applied to various models and tasks. We also propose a self-supervised pruning method that can be applied without the labeled data. Our experiments with several NLP tasks demonstrate the ability of TextPruner to reduce the model size without re-training the model.
