Robust extreme learning machine

详细信息    查看全文

• 作者：Punyaphol Horata ; ^{punhor1@kku.ac.th}Author Vitae ; Sirapat Chiewchanwattana ^{sunkra@kku.ac.th}Author Vitae ; Khamron Sunat ^{khamron_sunat@yahoo.com}Author Vitae
• 关键词：Extreme learning machine ; Moore&ndash ; Penrose pseudo inverse ; Singular Value Decomposition ; Extended Complete Orthogonal Decomposition ; Iteratively reweighted least squares ; Multivariate least-trimmed squares ; Minimax probability machine regression ; Meta-
• 刊名：Neurocomputing
• 出版年：2013
• 出版时间：15 February, 2013
• 年：2013
• 卷：102
• 期：Complete
• 页码：31-44
• 全文大小：628 K

文摘

The output weights computing of extreme learning machine (ELM) encounters two problems, the computational and outlier robustness problems. The computational problem occurs when the hidden layer output matrix is a not full column rank matrix or an ill-conditioned matrix because of randomly generated input weights and biases. An existing solution to this problem is Singular Value Decomposition (SVD) method. However, the training speed is still affected by the large complexity of SVD when computing the Moore-Penrose (MP) pseudo inverse. The outlier robustness problem may occur when the training data set contaminated with outliers then the accuracy rate of ELM is extremely affected. This paper proposes the Extended Complete Orthogonal Decomposition (ECOD) method to solve the computational problem in ELM weights computing via ECODLS algorithm. And the paper also proposes the other three algorithms, i.e. the iteratively reweighted least squares (IRWLS-ELM), ELM based on the multivariate least-trimmed squares (MLTS-ELM), and ELM based on the one-step reweighted MLTS (RMLTS-ELM) to solve the outlier robustness problem. However, they also encounter the computational problem. Therefore, the ECOD via ECODLS algorithm is also used successfully in the three proposed algorithms. The experiments of regression problems were conducted on both toy and real-world data sets. The outlier types are one-sided and two-sided outliers. Each experiment was randomly contaminated with outliers, of one type only, with 10 % , 20 % , 30 % , 40 % , and 50 % of the total training data size. Meta-metrics evaluation was used to measure the outlier robustness of the proposed algorithms compared to the existing algorithms, i.e. the minimax probability machine regression (MPMR) and the ordinary ELM. The experimental results showed that ECOD can effectively replace SVD. The ECOD is robust to the not full column rank or the ill-conditional problem. The speed of the ELM training using ECOD is also faster than the ordinary training algorithm. Moreover, the meta-metrics measure showed that the proposed algorithms are less affected by the increasing number of outliers than the existing algorithms.