论文标题
通过共同信息正规分配的无监督视觉表示学习
Unsupervised Visual Representation Learning via Mutual Information Regularized Assignment
论文作者
论文摘要
本文提出了共同信息正则分配（MIRA），这是一种伪标记的算法，用于受到信息最大化启发的无监督表示学习。我们将在线伪标记作为优化问题，以找到伪标记，以最大程度地提高标签和数据之间的互信息，同时接近给定的模型概率。我们得出了定点迭代方法，并证明了其收敛到最佳解决方案。与基准相反，MIRA与伪标记的预测相结合，可以实现一个简单而有效的基于聚类的表示的学习，而无需结合额外的培训技术或人工限制因素，例如采样策略，等电位约束等。相对较小的培训时期，Mira Achieves在Mira Achieves中所学到的代表性在各种下进行了局部的范围，包括各种局限性的任务和knn NN，包括linsears/k-nn。特别是，只有400个时代，我们的方法应用于具有Resnet-50体系结构的Imagenet数据集，可实现75.6％的线性评估精度。
This paper proposes Mutual Information Regularized Assignment (MIRA), a pseudo-labeling algorithm for unsupervised representation learning inspired by information maximization. We formulate online pseudo-labeling as an optimization problem to find pseudo-labels that maximize the mutual information between the label and data while being close to a given model probability. We derive a fixed-point iteration method and prove its convergence to the optimal solution. In contrast to baselines, MIRA combined with pseudo-label prediction enables a simple yet effective clustering-based representation learning without incorporating extra training techniques or artificial constraints such as sampling strategy, equipartition constraints, etc. With relatively small training epochs, representation learned by MIRA achieves state-of-the-art performance on various downstream tasks, including the linear/k-NN evaluation and transfer learning. Especially, with only 400 epochs, our method applied to ImageNet dataset with ResNet-50 architecture achieves 75.6% linear evaluation accuracy.
