论文标题
突触动态实现一阶自适应学习和体重对称性
Synaptic Dynamics Realize First-order Adaptive Learning and Weight Symmetry
论文作者
论文摘要
基于梯度的一阶自适应优化方法（例如ADAM优化器）在训练人工网络中很普遍，从而实现了最新的结果。这项工作试图回答以下问题，对于生物神经系统采用这种优化方法是否可行。为此，我们使用突触中的生物学上的机制来证明ADAM优化器的实现。提出的学习规则具有清晰的生物学对应关系，及时连续运行，并达到可比的亚当的表现。此外，我们提出了一种新方法，灵感来自神经科学中观察到的突触的易感性，以避免重量转运问题（BP）中体重传输问题的生物学不可能。只有本地信息和没有单独的训练阶段，此方法可以在前进和向后的信号通路中建立和维持重量对称性，并且适用于提议的生物学上合理的ADAM学习规则。这些机制可能会阐明生物突触动力学促进学习的方式。
Gradient-based first-order adaptive optimization methods such as the Adam optimizer are prevalent in training artificial networks, achieving the state-of-the-art results. This work attempts to answer the question whether it is viable for biological neural systems to adopt such optimization methods. To this end, we demonstrate a realization of the Adam optimizer using biologically-plausible mechanisms in synapses. The proposed learning rule has clear biological correspondence, runs continuously in time, and achieves performance to comparable Adam's. In addition, we present a new approach, inspired by the predisposition property of synapses observed in neuroscience, to circumvent the biological implausibility of the weight transport problem in backpropagation (BP). With only local information and no separate training phases, this method establishes and maintains weight symmetry in the forward and backward signaling paths, and is applicable to the proposed biologically plausible Adam learning rule. These mechanisms may shed light on the way in which biological synaptic dynamics facilitate learning.
