论文标题
Simsiam如何避免而没有负样本的崩溃？通过自我监督的对比学习的统一理解
How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning
论文作者
论文摘要
为了避免自我监督学习（SSL）崩溃，对比损失被广泛使用，但通常需要大量的负样本。如果没有负面样本尚未达到竞争性能，最近的一项工作引起了极大的关注，因为它提供了简约的简单暹罗（Simsiam）方法来避免崩溃。但是，如何避免没有负样本的崩溃的原因尚不完全清楚，我们的调查首先要重新审视原始Simsiam中的解释性主张。在驳斥了他们的主张后，我们引入了矢量分解，以根据$ l_2 $ normalized代表矢量的梯度分析来分析崩溃。这对消极样本和类似方式减轻崩溃的方式产生了统一的观点。这样的统一观点是为了理解SSL的最新进展而及时出现。
To avoid collapse in self-supervised learning (SSL), a contrastive loss is widely used but often requires a large number of negative samples. Without negative samples yet achieving competitive performance, a recent work has attracted significant attention for providing a minimalist simple Siamese (SimSiam) method to avoid collapse. However, the reason for how it avoids collapse without negative samples remains not fully clear and our investigation starts by revisiting the explanatory claims in the original SimSiam. After refuting their claims, we introduce vector decomposition for analyzing the collapse based on the gradient analysis of the $l_2$-normalized representation vector. This yields a unified perspective on how negative samples and SimSiam alleviate collapse. Such a unified perspective comes timely for understanding the recent progress in SSL.
