论文标题
MICSE：低声句子嵌入的相互信息对比度学习
miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddings
论文作者
论文摘要
本文介绍了MICSE，这是一种基于信息的对比学习框架，可在几个句子嵌入中显着提高最先进的作用。所提出的方法在对比度学习过程中强加了不同观点的注意力模式之间的一致性。学习用MICSE嵌入的句子嵌入需要每个句子的增强视图中实现结构一致性，从而使对比的自我监督学习更加有效。结果，提出的方法在几个射击学习领域中表现出强劲的表现。尽管与几次学习中的多个基准测试中的最新方法相比，它取得了优越的结果，但在完整的方案中是可比的。这项研究为有效的自我监督学习方法开辟了途径，这些学习方法比当前的对比方法更强大。
This paper presents miCSE, a mutual information-based contrastive learning framework that significantly advances the state-of-the-art in few-shot sentence embedding. The proposed approach imposes alignment between the attention pattern of different views during contrastive learning. Learning sentence embeddings with miCSE entails enforcing the structural consistency across augmented views for every sentence, making contrastive self-supervised learning more sample efficient. As a result, the proposed approach shows strong performance in the few-shot learning domain. While it achieves superior results compared to state-of-the-art methods on multiple benchmarks in few-shot learning, it is comparable in the full-shot scenario. This study opens up avenues for efficient self-supervised learning methods that are more robust than current contrastive methods for sentence embedding.
