论文标题
解决具有基于潜在的归一化的尖峰Q网络中的尖峰特征信息消失的问题
Solving the Spike Feature Information Vanishing Problem in Spiking Deep Q Network with Potential Based Normalization
论文作者
论文摘要
脑启发的尖峰神经网络（SNN）已成功应用于许多模式识别域。基于SNN的深层结构在感知任务（例如图像分类，目标检测）中取得了可观的结果。但是，深SNN在加强学习（RL）任务中的应用仍然是一个问题。尽管以前已经对SNN和RL的组合进行了研究，但其中大多数专注于浅网络的机器人控制问题，或使用ANN-SNN转换方法来实施Spiking Spiking Deep Q Network（SDQN）。在这项工作中，我们数学分析了SDQN中尖峰信号特征消失的问题，并提出了一种基于潜在的层归一化方法（PBLN）方法，以直接训练尖峰深度Q网络。实验表明，与最先进的ANN-SNN转换方法和其他SDQN作品相比，提议的PBLN Spiking Deep Q Networks（PL-SDQN）在Atari游戏任务上取得了更好的性能。
Brain inspired spiking neural networks (SNNs) have been successfully applied to many pattern recognition domains. The SNNs based deep structure have achieved considerable results in perceptual tasks, such as image classification, target detection. However, the application of deep SNNs in reinforcement learning (RL) tasks is still a problem to be explored. Although there have been previous studies on the combination of SNNs and RL, most of them focus on robotic control problems with shallow networks or using ANN-SNN conversion method to implement spiking deep Q Network (SDQN). In this work, we mathematically analyzed the problem of the disappearance of spiking signal features in SDQN and proposed a potential based layer normalization(pbLN) method to directly train spiking deep Q networks. Experiment shows that compared with state-of-art ANN-SNN conversion method and other SDQN works, the proposed pbLN spiking deep Q networks (PL-SDQN) achieved better performance on Atari game tasks.
