论文标题
LSTMS中语法依赖性的归因分析
Attribution Analysis of Grammatical Dependencies in LSTMs
论文作者
论文摘要
LSTM语言模型已被证明可以捕获对语法敏感的语法依赖性，例如主体 - 动词一致性，具有高度的准确性（Linzen等，2016，Inter Inter Elia）。但是，关于他们是否使用虚假相关性，还是真正能够与主体匹配的动词，仍然存在问题。本文主张了后一种假设。使用层次相关性传播（Bach等，2015），这是一种量化输入特征对模型行为的贡献的技术，我们表明数字一致性上的LSTM性能与模型的能力直接相关，该模型的能力将受试者与其他名词区分开。我们的结果表明，LSTM语言模型能够推断出句法依赖性的强大表示。
LSTM language models have been shown to capture syntax-sensitive grammatical dependencies such as subject-verb agreement with a high degree of accuracy (Linzen et al., 2016, inter alia). However, questions remain regarding whether they do so using spurious correlations, or whether they are truly able to match verbs with their subjects. This paper argues for the latter hypothesis. Using layer-wise relevance propagation (Bach et al., 2015), a technique that quantifies the contributions of input features to model behavior, we show that LSTM performance on number agreement is directly correlated with the model's ability to distinguish subjects from other nouns. Our results suggest that LSTM language models are able to infer robust representations of syntactic dependencies.
