Deep extreme learning machines: supervised autoencoding architecture for classification

详细信息    查看全文

• 作者：Migel D. Tissera ^{migel.tissera@mymail.unisa.edu.au" class="auth_mail" title="E-mail the corresponding author}Author Vitae ; Mark D. McDonnellAuthor Vitae
• 关键词：Extreme learning machine ; Supervised learning ; Autoencoder ; Classifier ; MNIST ; Deep neural network
• 刊名：Neurocomputing
• 出版年：2016
• 出版时间：22 January 2016
• 年：2016
• 卷：174
• 期：part_PA
• 页码：42-49
• 全文大小：633 K

文摘

We present a method for synthesising deep neural networks using Extreme Learning Machines (ELMs) as a stack of supervised autoencoders. We test the method using standard benchmark datasets for multi-class image classification (MNIST, CIFAR-10 and Google Streetview House Numbers (SVHN)), and show that the classification error rate can progressively improve with the inclusion of additional autoencoding ELM modules in a stack. Moreover, we found that the method can correctly classify up to 99.19% of MNIST test images, which surpasses the best error rates reported for standard 3-layer ELMs or previous deep ELM approaches when applied to MNIST. The approach simultaneously offers a significantly faster training algorithm to achieve its best performance (in the order of 5 min on a four-core CPU for MNIST) relative to a single ELM with the same total number of hidden units as the deep ELM, hence offering the best of both worlds: lower error rates and fast implementation.