论文标题
神经网络与数据无关的结构化修剪
Data-Independent Structured Pruning of Neural Networks via Coresets
论文作者
论文摘要
模型压缩对于在计算和内存资源有限的设备上部署神经网络至关重要。许多不同的方法表现出压缩模型和相似压缩率的可比精度。但是，大多数压缩方法都是基于启发式方法，并且在压缩率和任意新样本的近似误差之间的权衡方面没有最坏的保证。我们提出了第一个有效的结构性修剪算法，其压缩率与任何未来测试样本的近似误差之间有可证明的权衡。我们的方法基于核心框架，它通过上一层中的神经元/过滤器的核心近似于神经元/过滤器的输出，并丢弃其余的。我们以从底部到顶部以逐层方式应用此框架。与以前的作品不同，我们的核心是独立于数据的，这意味着它可以保证任何输入$ x \ in \ mathbb {r}^d $，包括对抗性的任何输入$ x \。
Model compression is crucial for deployment of neural networks on devices with limited computational and memory resources. Many different methods show comparable accuracy of the compressed model and similar compression rates. However, the majority of the compression methods are based on heuristics and offer no worst-case guarantees on the trade-off between the compression rate and the approximation error for an arbitrarily new sample. We propose the first efficient structured pruning algorithm with a provable trade-off between its compression rate and the approximation error for any future test sample. Our method is based on the coreset framework and it approximates the output of a layer of neurons/filters by a coreset of neurons/filters in the previous layer and discards the rest. We apply this framework in a layer-by-layer fashion from the bottom to the top. Unlike previous works, our coreset is data independent, meaning that it provably guarantees the accuracy of the function for any input $x\in \mathbb{R}^d$, including an adversarial one.
