论文标题
根据伯特的普遍依赖性：更具体，更一般
Universal Dependencies according to BERT: both more specific and more general
论文作者
论文摘要
这项工作着重于分析通过从自我攻击中提取标记的依赖树来捕获的句法抽象的形式和程度。 先前的工作表明，单个BERT头倾向于编码特定的依赖关系类型。我们通过将BERT关系与通用依赖关系（UD）注释进行明确比较来扩展这些发现，表明它们通常与一对一不匹配。 我们建议一种关系鉴定和句法树的结构的方法。我们的方法比以前的工作产生了更一致的依赖树，这表明它更好地解释了BERT中的句法抽象。同时，它只能在最少的监督下成功地应用，并在跨语言中概述。
This work focuses on analyzing the form and extent of syntactic abstraction captured by BERT by extracting labeled dependency trees from self-attentions. Previous work showed that individual BERT heads tend to encode particular dependency relation types. We extend these findings by explicitly comparing BERT relations to Universal Dependencies (UD) annotations, showing that they often do not match one-to-one. We suggest a method for relation identification and syntactic tree construction. Our approach produces significantly more consistent dependency trees than previous work, showing that it better explains the syntactic abstractions in BERT. At the same time, it can be successfully applied with only a minimal amount of supervision and generalizes well across languages.
