论文标题
测试时间批发归一化
Test-time Batch Normalization
论文作者
论文摘要
深度神经网络通常会遭受训练和测试之间的数据分布变化，并且观察到批处理统计数据反映了转变。在本文中，针对减轻测试时间的分配变化，我们在培训过程中重新审视批归一化（BN），并揭示了有益于测试时间优化的两个关键见解：$（i）$ $（i）$使用$（ii）$（II）$（II）$使用数据集级别的统计量来实现强大的优化和优化的优化。根据这两个见解，我们提出了一种新型的测试时间BN层设计GPREBN，该设计在测试过程中通过最小化熵损失而进行了优化。我们验证方法对具有分布变化的两个典型设置的有效性，即域的概括和鲁棒性任务。我们的GPREBN显着提高了测试时间的性能并实现了最新的结果。
Deep neural networks often suffer the data distribution shift between training and testing, and the batch statistics are observed to reflect the shift. In this paper, targeting of alleviating distribution shift in test time, we revisit the batch normalization (BN) in the training process and reveals two key insights benefiting test-time optimization: $(i)$ preserving the same gradient backpropagation form as training, and $(ii)$ using dataset-level statistics for robust optimization and inference. Based on the two insights, we propose a novel test-time BN layer design, GpreBN, which is optimized during testing by minimizing Entropy loss. We verify the effectiveness of our method on two typical settings with distribution shift, i.e., domain generalization and robustness tasks. Our GpreBN significantly improves the test-time performance and achieves the state of the art results.
