论文标题
通过基于变压器的序列概率指导符号自然语言诱导
Guiding Symbolic Natural Language Grammar Induction via Transformer-Based Sequence Probabilities
论文作者
论文摘要
根据变压器神经网络语言模型分配给句子（以及可能更长的单词序列），提出了一种用于管理自然语言的句法规则的新颖方法，以指导象征性学习过程，例如聚类和规则诱导。这种方法利用了变压器中学习的语言知识，而没有任何提及其内部表示。因此，该技术很容易适应更强大的语言模型的连续外观。我们展示了我们提出的技术的概念证明示例，它使用它来指导从我们先前的研究中提取的无监督符号链接 - 格拉米疗法诱导方法。
A novel approach to automated learning of syntactic rules governing natural languages is proposed, based on using probabilities assigned to sentences (and potentially longer word sequences) by transformer neural network language models to guide symbolic learning processes like clustering and rule induction. This method exploits the learned linguistic knowledge in transformers, without any reference to their inner representations; hence, the technique is readily adaptable to the continuous appearance of more powerful language models. We show a proof-of-concept example of our proposed technique, using it to guide unsupervised symbolic link-grammar induction methods drawn from our prior research.
