论文标题
基于BERT的双重嵌入模型，用于中国成语预测
A BERT-based Dual Embedding Model for Chinese Idiom Prediction
论文作者
论文摘要
中国成语是通常源自古代故事的特殊固定短语，其含义通常是高度惯用和非构成的。中文的成语预测任务是从一组候选人的成语中选择正确的成语，给定一个空白的上下文。我们提出了一个基于BERT的双嵌入模型，以编码上下文单词以及学习成语的双重嵌入。具体而言，我们首先将每个候选成语的嵌入与上下文中的空白相对应的隐藏表示形式匹配。然后，我们将每个候选成语的嵌入方式与上下文中所有令牌的隐藏表示形式相匹配。我们进一步建议将两种单独的惯用嵌入方式用于两种匹配。最近发布的中国成语披肩测试数据集的实验表明，我们提出的方法的性能要比现有的最新状态更好。消融实验还表明，上下文集合和双重嵌入都有助于提高性能。
Chinese idioms are special fixed phrases usually derived from ancient stories, whose meanings are oftentimes highly idiomatic and non-compositional. The Chinese idiom prediction task is to select the correct idiom from a set of candidate idioms given a context with a blank. We propose a BERT-based dual embedding model to encode the contextual words as well as to learn dual embeddings of the idioms. Specifically, we first match the embedding of each candidate idiom with the hidden representation corresponding to the blank in the context. We then match the embedding of each candidate idiom with the hidden representations of all the tokens in the context thorough context pooling. We further propose to use two separate idiom embeddings for the two kinds of matching. Experiments on a recently released Chinese idiom cloze test dataset show that our proposed method performs better than the existing state of the art. Ablation experiments also show that both context pooling and dual embedding contribute to the improvement of performance.
