论文标题
关于深度神经网络中线性区域特性的经验研究
Empirical Studies on the Properties of Linear Regions in Deep Neural Networks
论文作者
论文摘要
带有分段线性激活的深神经网络（DNN）可以将输入空间划分为拟合不同线性函数的许多小线性区域。人们认为这些区域的数量代表DNN的表现力。本文提供了一种新颖而细致的视角来研究DNNS：我们不仅研究了线性区域的数量，还可以研究它们的局部特性，例如独奏器，相应的超平面，决策范围，决策边界的方向以及周围地区的相关性。我们从经验上观察到，不同的优化技术导致了完全不同的线性区域，即使它们产生了相似的分类精度。我们希望我们的研究能够激发新型优化技术的设计，并帮助发现和分析DNN的行为。
A deep neural network (DNN) with piecewise linear activations can partition the input space into numerous small linear regions, where different linear functions are fitted. It is believed that the number of these regions represents the expressivity of the DNN. This paper provides a novel and meticulous perspective to look into DNNs: Instead of just counting the number of the linear regions, we study their local properties, such as the inspheres, the directions of the corresponding hyperplanes, the decision boundaries, and the relevance of the surrounding regions. We empirically observed that different optimization techniques lead to completely different linear regions, even though they result in similar classification accuracies. We hope our study can inspire the design of novel optimization techniques, and help discover and analyze the behaviors of DNNs.
