论文标题
联合修剪：通过联合学习提高神经网络效率
Federated Pruning: Improving Neural Network Efficiency with Federated Learning
论文作者
论文摘要
自动语音识别模型需要大量的语音数据进行培训，并且此类数据的收集通常会导致隐私问题。联合学习已被广泛使用，并被认为是一种有效的分散技术，通过协作学习共享的预测模型，同时将数据保留在不同客户端设备上。但是，客户设备上有限的计算和通信资源给大型模型带来了实际困难。为了克服此类挑战，我们提出联合修剪以在联合环境下训练还原模型，同时与完整模型相比保持相似的性能。此外，与集中式培训相比，还可以利用大量客户数据来改善修剪结果。我们探索不同的修剪方案，并提供了我们方法有效性的经验证据。
Automatic Speech Recognition models require large amount of speech data for training, and the collection of such data often leads to privacy concerns. Federated learning has been widely used and is considered to be an effective decentralized technique by collaboratively learning a shared prediction model while keeping the data local on different clients devices. However, the limited computation and communication resources on clients devices present practical difficulties for large models. To overcome such challenges, we propose Federated Pruning to train a reduced model under the federated setting, while maintaining similar performance compared to the full model. Moreover, the vast amount of clients data can also be leveraged to improve the pruning results compared to centralized training. We explore different pruning schemes and provide empirical evidence of the effectiveness of our methods.
