论文标题
通过多任务质量检查在语言模型中灌输类型知识
Instilling Type Knowledge in Language Models via Multi-Task QA
论文作者
论文摘要
理解人类语言通常需要理解实体及其在知识分类法中的地位 - 他们的类型。以前学习实体类型的方法依赖于具有粗，嘈杂和不完整标签的数据集上的培训分类器。我们介绍了一种在语言模型中灌输精细类型知识的方法，并在利用知识库基础文档和知识图的文本到文本预培训中进行文本到文本培训。我们创建了Wikiwiki数据集：来自10m Wikipedia文章的实体和段落，这些文章链接到Wikidata知识图，带有41K类型。在Wikiwiki接受培训的模型在零摄像对话框状态跟踪基准中实现了最先进的性能，在Wikipedia文章中准确推断实体类型，并可以发现人类法官认为有用的新类型。
Understanding human language often necessitates understanding entities and their place in a taxonomy of knowledge -- their types. Previous methods to learn entity types rely on training classifiers on datasets with coarse, noisy, and incomplete labels. We introduce a method to instill fine-grained type knowledge in language models with text-to-text pre-training on type-centric questions leveraging knowledge base documents and knowledge graphs. We create the WikiWiki dataset: entities and passages from 10M Wikipedia articles linked to the Wikidata knowledge graph with 41K types. Models trained on WikiWiki achieve state-of-the-art performance in zero-shot dialog state tracking benchmarks, accurately infer entity types in Wikipedia articles, and can discover new types deemed useful by human judges.
