论文标题
HREL：基于激活图和类标签之间的高相关性过滤器修剪
HRel: Filter Pruning based on High Relevance between Activation Maps and Class Labels
论文作者
论文摘要
本文提出了一种基于信息瓶颈理论的过滤器修剪方法，该方法使用了一种称为互信息（MI）的统计措施。使用过滤器的激活图和注释计算过滤器和类标签之间的MI，也称为\ textit {相关}。具有高相关性（HREL）的过滤器被认为更重要。因此，将修剪与类标签具有较低相互信息的最小重要过滤器。与现有的基于MI的修剪方法不同，所提出的方法纯粹基于其相应的激活图与类标签的关系来确定过滤器的重要性。使用Lenet-5，VGG-16，Resnet-56 \ TextColor {MyBlue} {，Resnet-1110和Resnet-50等体系结构可证明对MNIST，CIFAR-10和Imagenet数据集的拟议修剪方法的功效。提出的方法显示了Lenet-5，VGG-16，Resnet-56，Resnet-110和Resnet-50体系结构的最新修剪结果。在实验中，我们修剪97.98 \％，84.85 \％，76.89 \％，76.95 \％\％和63.99 \％的浮点操作（FLOP）s（flop）S，来自LENET-5，VGG-16，VGG-16，VGG-16，RESNET-56，RESNET-56，RESNET-56，RESNET-1110，RESNET-1110，RESNET-50，prun prun prun prun prun prun prun prun prun prun the prun prun prun prun prun prun prun prun prun prun the prun prun prun。过滤器修剪方法。即使在巨大的LENET-5的卷积层中修剪过滤器（分别为20、50到2、3），也只观察到较小的精度下降0.52 \％。值得注意的是，对于VGG-16，94.98 \％参数减少，仅在TOP-1准确性下降0.36 \％。 \ textColor {myblue} {resnet-50在修剪66.42 \％的触发器后，前5个精度下降了1.17 \％。}除了修剪，信息平面的信息平面动力学的瓶颈理论还针对各种交流型神经网络架构进行了分析，并具有效果。
This paper proposes an Information Bottleneck theory based filter pruning method that uses a statistical measure called Mutual Information (MI). The MI between filters and class labels, also called \textit{Relevance}, is computed using the filter's activation maps and the annotations. The filters having High Relevance (HRel) are considered to be more important. Consequently, the least important filters, which have lower Mutual Information with the class labels, are pruned. Unlike the existing MI based pruning methods, the proposed method determines the significance of the filters purely based on their corresponding activation map's relationship with the class labels. Architectures such as LeNet-5, VGG-16, ResNet-56\textcolor{myblue}{, ResNet-110 and ResNet-50 are utilized to demonstrate the efficacy of the proposed pruning method over MNIST, CIFAR-10 and ImageNet datasets. The proposed method shows the state-of-the-art pruning results for LeNet-5, VGG-16, ResNet-56, ResNet-110 and ResNet-50 architectures. In the experiments, we prune 97.98 \%, 84.85 \%, 76.89\%, 76.95\%, and 63.99\% of Floating Point Operation (FLOP)s from LeNet-5, VGG-16, ResNet-56, ResNet-110, and ResNet-50 respectively.} The proposed HRel pruning method outperforms recent state-of-the-art filter pruning methods. Even after pruning the filters from convolutional layers of LeNet-5 drastically (i.e. from 20, 50 to 2, 3, respectively), only a small accuracy drop of 0.52\% is observed. Notably, for VGG-16, 94.98\% parameters are reduced, only with a drop of 0.36\% in top-1 accuracy. \textcolor{myblue}{ResNet-50 has shown a 1.17\% drop in the top-5 accuracy after pruning 66.42\% of the FLOPs.} In addition to pruning, the Information Plane dynamics of Information Bottleneck theory is analyzed for various Convolutional Neural Network architectures with the effect of pruning.
