论文标题
通过riemannian歧管进行保形改造：将特定于任务的图形蒸馏到预验证的嵌入中
Conformal retrofitting via Riemannian manifolds: distilling task-specific graphs into pretrained embeddings
论文作者
论文摘要
预处理的（语言）嵌入是通用的，任务无关的特征表示实体（如单词），这对于许多机器学习应用至关重要。这些表示可以通过翻新来丰富，这是一类方法，这些方法将特定于任务的域知识纳入了这些实体子集的图形。但是，现有的翻新算法面临两个局限性：它们通过未与缺失实体的关系而过分拟合观察到的图；他们仅通过学习欧几里得歧管中的嵌入来使观察到的图拟合，这些嵌入者甚至无法忠实地代表简单的树结构或循环图。我们通过两个主要贡献解决了这些问题：（i）我们提出了一个新颖的正规化器，一种综合性的正规化器，可以保留局部几何形状，该几何形状从预算的嵌入中 - 使其对缺失实体的概括和（ii）新的Riemannian前进层学习，该层学会学到以在非核核核核核核葡萄核的绘制中映射嵌入，以代表核核的均匀图表。通过在WordNet上的实验，我们证明了保组处正常化程序甚至可以防止现有的（仅欧几里得）方法过于拟合缺失的Entities的链路预测，以及---以及Riemannian feedforward layer---学习非欧盟的非欧盟式嵌入式，使它们胜过它们。
Pretrained (language) embeddings are versatile, task-agnostic feature representations of entities, like words, that are central to many machine learning applications. These representations can be enriched through retrofitting, a class of methods that incorporate task-specific domain knowledge encoded as a graph over a subset of these entities. However, existing retrofitting algorithms face two limitations: they overfit the observed graph by failing to represent relationships with missing entities; and they underfit the observed graph by only learning embeddings in Euclidean manifolds, which cannot faithfully represent even simple tree-structured or cyclic graphs. We address these problems with two key contributions: (i) we propose a novel regularizer, a conformality regularizer, that preserves local geometry from the pretrained embeddings---enabling generalization to missing entities and (ii) a new Riemannian feedforward layer that learns to map pre-trained embeddings onto a non-Euclidean manifold that can better represent the entire graph. Through experiments on WordNet, we demonstrate that the conformality regularizer prevents even existing (Euclidean-only) methods from overfitting on link prediction for missing entities, and---together with the Riemannian feedforward layer---learns non-Euclidean embeddings that outperform them.
