论文标题
在同步过渡边缘附近的最佳加固学习
Optimal reinforcement learning near the edge of synchronization transition
论文作者
论文摘要
最近的实验和理论研究表明，皮质动力学的推定临界性可能对应于同步相变。与标准吸收状态相变附近的临界行为相比，此类关键点附近的关键动力学需要进一步进行进一步的研究。由于由于存在同步神经网络的同步转变边缘的学习和自组织临界（SOC）的现象，由于存在峰值依赖性可塑性（STDP），因此很容易询问：在神经网络中同步和学习之间的关系是什么？此外，学习是否从同步过渡边缘的SOC中受益？在本文中，我们打算解决这些重要问题。因此，我们构建了一个自主认知系统的生物学启发模型，该模型学会了执行刺激反应任务。我们使用通过多巴胺调制的STDP实现的增强学习规则来训练该系统。我们发现，该系统在增加平均轴突时间延迟时表现出从同步到异步神经振荡的连续过渡。我们表征了系统的学习性能，并观察到它是在同步转换附近进行了优化的。我们还研究了系统中的神经元雪崩，并提供了证据表明，以稍微临界的状态实现了优化的学习。
Recent experimental and theoretical studies have indicated that the putative criticality of cortical dynamics may corresponds to a synchronization phase transition. The critical dynamics near such a critical point needs further investigation specifically when compared to the critical behavior near the standard absorbing state phase transition. Since the phenomena of learning and self-organized criticality (SOC) at the edge of synchronization transition can emerge jointly in spiking neural networks due to the presence of spike-timing dependent plasticity (STDP), it is tempting to ask: What is the relationship between synchronization and learning in neural networks? Further, does learning benefit from SOC at the edge of synchronization transition? In this paper, we intend to address these important issues. Accordingly, we construct a biologically inspired model of an autonomous cognitive system which learns to perform stimulus-response tasks. We train this system using a reinforcement learning rule implemented through dopamine-modulated STDP. We find that the system exhibits a continuous transition from synchronous to asynchronous neural oscillations upon increasing the average axonal time delay. We characterize the learning performance of the system and observe that it is optimized near the synchronization transition. We also study neuronal avalanches in the system and provide evidence that optimized learning is achieved in a slightly supercritical state.
