论文标题
通过在线扩散批处理歧管上的自我介绍改善深度度量学习
Self-distillation with Online Diffusion on Batch Manifolds Improves Deep Metric Learning
论文作者
论文摘要
最近的深度度量学习（DML）方法通常仅利用类标签，以使阳性样本远离负面样本。但是，这种类型的方法通常会忽略数据中隐藏的关键知识（例如，类信息变化），这对训练有素的模型的概括有害。为了减轻这个问题，在本文中，我们建议针对DML的在线批次扩散自distillation（OBD-SD）。具体而言，我们首先提出了一个简单但有效的渐进自我散步（PSD），该自我验证（PSD）在训练过程中逐渐从模型本身中提取知识。 PSD实现的软距离目标可以在样本之间提供更丰富的关系信息，这对嵌入表示的多样性有益。然后，我们使用在线批处理扩散过程（OBDP）扩展PSD，该过程是捕获每个批处理中歧管的局部几何结构，以便它可以揭示批处理中样本之间的内在关系并产生更好的软距离目标。请注意，我们的OBDP能够恢复原始PSD获得的不足的流形关系并实现显着的性能提高。我们的OBD-SD是一个灵活的框架，可以集成到最新的（SOTA）DML方法中。对各种基准测试，即Cub200，CARS196和Stanford Online Products进行的广泛实验表明，我们的OBD-SD始终在多个数据集中持续提高了具有可忽略的额外培训时间，从而提高了现有DML方法的性能，从而实现了非常有竞争力的结果。代码：\ url {https://github.com/zelongzeng/obd-sd_pytorch}
Recent deep metric learning (DML) methods typically leverage solely class labels to keep positive samples far away from negative ones. However, this type of method normally ignores the crucial knowledge hidden in the data (e.g., intra-class information variation), which is harmful to the generalization of the trained model. To alleviate this problem, in this paper we propose Online Batch Diffusion-based Self-Distillation (OBD-SD) for DML. Specifically, we first propose a simple but effective Progressive Self-Distillation (PSD), which distills the knowledge progressively from the model itself during training. The soft distance targets achieved by PSD can present richer relational information among samples, which is beneficial for the diversity of embedding representations. Then, we extend PSD with an Online Batch Diffusion Process (OBDP), which is to capture the local geometric structure of manifolds in each batch, so that it can reveal the intrinsic relationships among samples in the batch and produce better soft distance targets. Note that our OBDP is able to restore the insufficient manifold relationships obtained by the original PSD and achieve significant performance improvement. Our OBD-SD is a flexible framework that can be integrated into state-of-the-art (SOTA) DML methods. Extensive experiments on various benchmarks, namely CUB200, CARS196, and Stanford Online Products, demonstrate that our OBD-SD consistently improves the performance of the existing DML methods on multiple datasets with negligible additional training time, achieving very competitive results. Code: \url{https://github.com/ZelongZeng/OBD-SD_Pytorch}
