论文标题
在弥合均值随机神经网络中平均宽度和有限宽度之间的差距时，批处理
On Bridging the Gap between Mean Field and Finite Width in Deep Random Neural Networks with Batch Normalization
论文作者
论文摘要
平均场理论被广泛用于神经网络的理论研究中。在本文中，我们分析了深度预测浓度的作用，特别是针对初始化时批准（BN）的深层多层感知器（MLP）。通过将网络宽度缩放到Infinity，可以假定平均场预测会遭受层面误差的影响。我们证明，BN可以稳定避免平均场预测误差传播的表示的分布。这种以几何混合特性为特征的稳定化使我们能够在具有有限宽度的无限深度神经网络中建立浓度界限。
Mean field theory is widely used in the theoretical studies of neural networks. In this paper, we analyze the role of depth in the concentration of mean-field predictions, specifically for deep multilayer perceptron (MLP) with batch normalization (BN) at initialization. By scaling the network width to infinity, it is postulated that the mean-field predictions suffer from layer-wise errors that amplify with depth. We demonstrate that BN stabilizes the distribution of representations that avoids the error propagation of mean-field predictions. This stabilization, which is characterized by a geometric mixing property, allows us to establish concentration bounds for mean field predictions in infinitely-deep neural networks with a finite width.
