Robust learning in SpikeProp

详细信息    查看全文

• 作者：Sumit Bam Shrestha ^{bam_sumit@hotmail.com sumitbam001@e.ntu.edu.sg} ; Qing Song ; ^{eqsong@ntu.edu.sg}
• 关键词：Spiking Neural Network ; Error analysis ; Weight convergence ; Robust stability ; Supervised learning ; Adaptive learning rate
• 刊名：Neural Networks
• 出版年：2017
• 出版时间：February 2017
• 年：2017
• 卷：86
• 期：Complete
• 页码：54-68
• 全文大小：1783 K
• 卷排序：86

文摘

Training a Spiking Neural Network using SpikeProp and its derivatives faces stability issues. Surges, marked by a sudden rise in learning cost, are a common occurrence during the learning process. They disrupt the learning process and often destabilize the process resulting in failure. A proper learning rate, which is neither too small nor too big, is important to minimize surges. Furthermore, external disturbances due to imperfection in sample data as well as internal disturbances are additional destabilizing source during the learning process. In this paper, we perform error system analysis incorporating external disturbance, followed by weight convergence analysis along with detailed robust stability analysis of SpikeProp learning process to ensure error bound of the learning process. Based on these results, we propose a robust adaptive learning rate scheme that aligns with the results of theoretical analysis. The performance of the proposed method has been compared with other prevalent methods based on different benchmark datasets and the results demonstrate that our method indeed has better performance in terms of convergence and learning speed as well.