论文标题
正规化不变性以扩大数据可以改善监督学习
Regularising for invariance to data augmentation improves supervised learning
论文作者
论文摘要
数据增强用于机器学习，以使分类器不变性地标记具有扩展性转换。通常，仅通过在培训期间包括一个增强输入来鼓励这种不变性。但是，几项作品最近表明，每个输入使用多个增强量可以改善概括，也可以用来更明确地融入不向导。在这项工作中，我们首先从经验上比较了这些最近提出的目标，这些目标在它们依赖于明确或隐式正规化以及它们在哪个级别上编码不变的级别上有所不同。我们表明，与同一输入的不同增强相比，最佳性能方法的预测也是最相似的。受到这一观察的启发，我们提出了一个明确的正规机构，以鼓励对单个模型预测水平的不变性。通过对CIFAR-100和Imagenet的广泛实验，我们表明该显式正规机构（i）改善了概括，（ii）等于所有被考虑的目标之间的性能差异。我们的结果表明，促进神经网络水平不变的目标本身比通过平均非不同模型预测实现不变性的目标更具概括性。
Data augmentation is used in machine learning to make the classifier invariant to label-preserving transformations. Usually this invariance is only encouraged implicitly by including a single augmented input during training. However, several works have recently shown that using multiple augmentations per input can improve generalisation or can be used to incorporate invariances more explicitly. In this work, we first empirically compare these recently proposed objectives that differ in whether they rely on explicit or implicit regularisation and at what level of the predictor they encode the invariances. We show that the predictions of the best performing method are also the most similar when compared on different augmentations of the same input. Inspired by this observation, we propose an explicit regulariser that encourages this invariance on the level of individual model predictions. Through extensive experiments on CIFAR-100 and ImageNet we show that this explicit regulariser (i) improves generalisation and (ii) equalises performance differences between all considered objectives. Our results suggest that objectives that encourage invariance on the level of the neural network itself generalise better than those that achieve invariance by averaging predictions of non-invariant models.
