论文标题
HyperDecoder：多任务NLP的实例特定解码器
Hyperdecoders: Instance-specific decoders for multi-task NLP
论文作者
论文摘要
我们研究了用于NLP中多任务处理的输入条件的超网络，从而为解码器生成了使用以编码器输出为条件的超网络的参数改编。这种方法为每个输入实例产生独特的解码器适应，使网络具有比仅在每个任务中产生一个解码器适应的先前工作更大的灵活性。我们将我们的方法应用于序列分类任务，提取质量检查和汇总，发现它超过了先前的参数有效微调方法，并且通常超过对基本模型完全挑战的表现。对我们超网络使用的嵌入的分析表明，它们对输出标签和类型敏感，这表明我们的方法从编码器表示到输出标签更好。我们的代码可在https://github.com/allenai/hyperdecoders上公开获取。
We investigate input-conditioned hypernetworks for multi-tasking in NLP, generating parameter-efficient adaptations for a decoder using a hypernetwork conditioned on the output of an encoder. This approach produces a unique decoder adaptation for every input instance, allowing the network a larger degree of flexibility than prior work that only produces one decoder adaptation per task. We apply our method to sequence classification tasks, extractive QA, and summarisation and find that it surpasses previous parameter efficient fine-tuning methods and often outperforms fully finetuning the underlying model. An analysis of the embeddings used by our hypernetwork shows that they are sensitive to output label and type, suggesting that our approach better maps from encoder representations to output labels. Our code is publicly available at https://github.com/allenai/hyperdecoders.
