论文标题
在有限的数据方案下，基于示范学习的鲁棒性
Robustness of Demonstration-based Learning Under Limited Data Scenario
论文作者
论文摘要
基于示范的学习表现出在有限的数据方案下刺激预验证的语言模型能力的巨大潜力。简单地通过一些演示来增强输入可以显着提高几杆NER的性能。但是，为什么这样的演示对学习过程有益，因为示范和预测之间没有明确的一致性，因此尚不清楚。在本文中，我们通过逐渐从标准信息中删除直觉上有用的信息来设计病理演示，以深入了解基于演示的序列标记的鲁棒性，并表明（1）由随机代币组成的演示仍然使模型使模型成为更好的几次学习者； （2）随机演示的长度和随机令牌的相关性是影响性能的主要因素； （3）演示增加了模型预测对捕获的浅表模式的置信度。我们已在https://github.com/salt-nlp/robustdemo上公开发布了代码。
Demonstration-based learning has shown great potential in stimulating pretrained language models' ability under limited data scenario. Simply augmenting the input with some demonstrations can significantly improve performance on few-shot NER. However, why such demonstrations are beneficial for the learning process remains unclear since there is no explicit alignment between the demonstrations and the predictions. In this paper, we design pathological demonstrations by gradually removing intuitively useful information from the standard ones to take a deep dive of the robustness of demonstration-based sequence labeling and show that (1) demonstrations composed of random tokens still make the model a better few-shot learner; (2) the length of random demonstrations and the relevance of random tokens are the main factors affecting the performance; (3) demonstrations increase the confidence of model predictions on captured superficial patterns. We have publicly released our code at https://github.com/SALT-NLP/RobustDemo.
