论文标题
使用不平衡的激活分布提高二元神经网络的准确性
Improving Accuracy of Binary Neural Networks using Unbalanced Activation Distribution
论文作者
论文摘要
神经网络模型的二进化被认为是在资源受限环境（例如移动设备）上部署深层神经网络模型的有前途方法之一。但是，与全精确的对应模型相比，二元神经网络（BNN）往往会遭受严重的准确性降解。提出了几种技术来提高BNN的准确性。一种方法是平衡二进制激活的分布，以使二进制激活中的信息量最大。基于广泛的分析，与以前的工作形成鲜明对比，我们认为不平衡的激活分布实际上可以提高BNN的准确性。我们还表明，调整二进制激活函数的阈值导致二进制激活的分布不平衡，从而提高了BNN模型的准确性。实验结果表明，可以通过简单地移动二进制激活函数的阈值而无需任何其他修改，可以提高先前BNN模型（例如Xnor-NET和BI-REAL-NET）的准确性。
Binarization of neural network models is considered as one of the promising methods to deploy deep neural network models on resource-constrained environments such as mobile devices. However, Binary Neural Networks (BNNs) tend to suffer from severe accuracy degradation compared to the full-precision counterpart model. Several techniques were proposed to improve the accuracy of BNNs. One of the approaches is to balance the distribution of binary activations so that the amount of information in the binary activations becomes maximum. Based on extensive analysis, in stark contrast to previous work, we argue that unbalanced activation distribution can actually improve the accuracy of BNNs. We also show that adjusting the threshold values of binary activation functions results in the unbalanced distribution of the binary activation, which increases the accuracy of BNN models. Experimental results show that the accuracy of previous BNN models (e.g. XNOR-Net and Bi-Real-Net) can be improved by simply shifting the threshold values of binary activation functions without requiring any other modification.
