论文标题
对比度学习可以为大约观看不变的功能找到最佳基础
Contrastive Learning Can Find An Optimal Basis For Approximately View-Invariant Functions
论文作者
论文摘要
对比学习是学习自我监督的表述的有力框架，可以很好地推广到下游监督任务。我们表明，可以将多种现有的对比学习方法重新解释为近似固定阳性对内核的学习内核函数。然后，我们证明，在正面假设正面假设正面的标签上，通过将该内核与PCA相结合，可以最大程度地减少线性预测变量最差的近似近似误差。我们的分析是基于目标函数的分解，该函数是基于正对马尔可夫链的征函数的，以及这些特征函数与内核PCA的输出之间令人惊讶的等效性。我们使用内核PCA表示为下游线性预测提供了概括，并在一系列合成任务上表现出，将核PCA应用于对比度学习模型确实可以近似地恢复Markov Chain feenfunctions，尽管精度取决于kernel参数化以及增强强度。
Contrastive learning is a powerful framework for learning self-supervised representations that generalize well to downstream supervised tasks. We show that multiple existing contrastive learning methods can be reinterpreted as learning kernel functions that approximate a fixed positive-pair kernel. We then prove that a simple representation obtained by combining this kernel with PCA provably minimizes the worst-case approximation error of linear predictors, under a straightforward assumption that positive pairs have similar labels. Our analysis is based on a decomposition of the target function in terms of the eigenfunctions of a positive-pair Markov chain, and a surprising equivalence between these eigenfunctions and the output of Kernel PCA. We give generalization bounds for downstream linear prediction using our Kernel PCA representation, and show empirically on a set of synthetic tasks that applying Kernel PCA to contrastive learning models can indeed approximately recover the Markov chain eigenfunctions, although the accuracy depends on the kernel parameterization as well as on the augmentation strength.
