论文标题
使用噪声来探测复发性神经网络结构和修剪突触
Using noise to probe recurrent neural network structure and prune synapses
论文作者
论文摘要
大脑中的许多网络都稀少，大脑在开发和学习过程中消除了突触。大脑如何决定修剪哪些突触？在经常性的网络中，确定两个神经元之间突触的重要性是一个困难的计算问题，这取决于神经元在它们之间的所有可能信息流中扮演的作用。噪声在神经系统中无处不在，通常被认为是要克服的刺激性。在这里，我们建议噪声可以在突触修剪中发挥功能作用，从而使大脑能够探测网络结构并确定哪些突触是冗余的。我们构建了一个简单的，本地的，无监督的可塑性规则，该规则仅使用突触重量和相邻神经元的噪声驱动的协方差来增强或李子突触。对于线性和整流线性网络的子集，我们证明该规则保留了原始矩阵的频谱，即使在渐进式的突触接近的平触差异时，也可以保留网络动力学1。可塑性规则在生物学上是可行的，并且可能暗示在Neural Computation中发挥新作用。
Many networks in the brain are sparsely connected, and the brain eliminates synapses during development and learning. How could the brain decide which synapses to prune? In a recurrent network, determining the importance of a synapse between two neurons is a difficult computational problem, depending on the role that both neurons play and on all possible pathways of information flow between them. Noise is ubiquitous in neural systems, and often considered an irritant to be overcome. Here we suggest that noise could play a functional role in synaptic pruning, allowing the brain to probe network structure and determine which synapses are redundant. We construct a simple, local, unsupervised plasticity rule that either strengthens or prunes synapses using only synaptic weight and the noise-driven covariance of the neighboring neurons. For a subset of linear and rectified-linear networks, we prove that this rule preserves the spectrum of the original matrix and hence preserves network dynamics even when the fraction of pruned synapses asymptotically approaches 1. The plasticity rule is biologically-plausible and may suggest a new role for noise in neural computation.
