论文标题
联合学习主题和特定于主题的单词嵌入的神经生成模型
A Neural Generative Model for Joint Learning Topics and Topic-Specific Word Embeddings
论文作者
论文摘要
我们提出了一个新颖的生成模型，以探索联合学习主题和特定于主题的单词嵌入的本地和全球环境。特别是，我们假设全局潜在主题是在文档中共享的，一个单词是由一个隐藏的语义向量编码其上下文语义含义生成的，并且其上下文单词是在隐藏的语义向量和全局潜在主题上生成的。主题是用嵌入一词共同培训的。训练有素的模型将单词映射到主题依赖性的嵌入式，该嵌入自然地解决了单词polysemy的问题。实验结果表明，所提出的模型在单词相似性评估和单词sense否认中都优于单词级别的嵌入方法。此外，与现有的神经主题模型或其他用于主题和单词嵌入的联合学习模型相比，该模型还提取了更连贯的主题。最后，该模型可以轻松地与现有的深层上下文化嵌入学习方法集成，以进一步改善下游任务（例如情感分类）的性能。
We propose a novel generative model to explore both local and global context for joint learning topics and topic-specific word embeddings. In particular, we assume that global latent topics are shared across documents, a word is generated by a hidden semantic vector encoding its contextual semantic meaning, and its context words are generated conditional on both the hidden semantic vector and global latent topics. Topics are trained jointly with the word embeddings. The trained model maps words to topic-dependent embeddings, which naturally addresses the issue of word polysemy. Experimental results show that the proposed model outperforms the word-level embedding methods in both word similarity evaluation and word sense disambiguation. Furthermore, the model also extracts more coherent topics compared with existing neural topic models or other models for joint learning of topics and word embeddings. Finally, the model can be easily integrated with existing deep contextualized word embedding learning methods to further improve the performance of downstream tasks such as sentiment classification.
