论文标题
Distilcamembert：蒸馏法国模型Camembert
DistilCamemBERT: a distillation of the French model CamemBERT
论文作者
论文摘要
基于变压器结构的现代自然语言处理（NLP）模型在非常多样化的任务方面代表了最新技术。但是，这些模型很复杂，代表其中最小的几亿个参数。这可能会阻碍他们在工业层面的采用，因此很难扩大到合理的基础设施和/或遵守社会和环境责任。为此，我们在本文中介绍了一个模型，该模型大大降低了著名法国模型（Camembert）的计算成本，同时保持良好的性能。
Modern Natural Language Processing (NLP) models based on Transformer structures represent the state of the art in terms of performance on very diverse tasks. However, these models are complex and represent several hundred million parameters for the smallest of them. This may hinder their adoption at the industrial level, making it difficult to scale up to a reasonable infrastructure and/or to comply with societal and environmental responsibilities. To this end, we present in this paper a model that drastically reduces the computational cost of a well-known French model (CamemBERT), while preserving good performance.
