论文标题
伯特真的同意吗？对句法任务的词汇依赖的细粒度分析
Does BERT really agree ? Fine-grained Analysis of Lexical Dependence on a Syntactic Task
论文作者
论文摘要
尽管基于变压器的神经语言模型在各种任务上表现出令人印象深刻的表现，但其概括能力尚未得到充分理解。他们已被证明在各种各样的设置中都可以在主题 - 动力数量协议上表现出色，这表明即使没有明确的监督，他们也学会了在训练过程中跟踪句法依赖性。在本文中，我们研究了BERT能够在靶向句法模板上执行词汇独立的主体数量一致性（NA）的程度。为此，在对伯特行为的新细粒分析中，我们破坏了每种靶向结构的天然发生刺激中发现的词汇模式。我们对非CE句子的结果表明，对于简单的模板，模型可以很好地推广，但是当存在一个吸引子时，该模型无法执行词汇独立的句法概括。
Although transformer-based Neural Language Models demonstrate impressive performance on a variety of tasks, their generalization abilities are not well understood. They have been shown to perform strongly on subject-verb number agreement in a wide array of settings, suggesting that they learned to track syntactic dependencies during their training even without explicit supervision. In this paper, we examine the extent to which BERT is able to perform lexically-independent subject-verb number agreement (NA) on targeted syntactic templates. To do so, we disrupt the lexical patterns found in naturally occurring stimuli for each targeted structure in a novel fine-grained analysis of BERT's behavior. Our results on nonce sentences suggest that the model generalizes well for simple templates, but fails to perform lexically-independent syntactic generalization when as little as one attractor is present.
