多任务正则极限学习机的研究与应用
|
摘要
MT-ELM通过隐含层共享不同任务间的数据特性实现多任务学习,但MT-ELM忽略任务间关联程度的差异以及存在的过拟合问题,为此提出基于MT-RELM软测量建模方法。首先,利用RELM解决过拟合问题;其次,考虑任务之间关联度的差异,基于相关性较强的任务其权值向量也较相似的假设,在每个任务输出权值的基础上加入约束条件,利用此约束条件表示任务间的相关程度;最后,利用ADMM算法迭代求解得到MT-RELM的模型参数。基于合成数据集与湿式球磨机数据集的结果表明,此算法可有效地提高模型的预测精度以及泛化能力。
MT-ELM enables multi-task learning using hidden layers to share data characteristics among different tasks.However, MT-ELM ignores correlation differences between tasks and over-fitting problem. Therefore, MT-RELM is proposed. Firstly, RELM is used to solve the over-fitting problem. Secondly, considering correlation differences between tasks, the constraints are added to output weights based on the assumptions that similar task having similar weight, so this paper uses this constraint to indicate relevance level between tasks. Finally, ADMM algorithm are used to solve MT-RELM model parameters. The results based on the synthetic dataset and wet ball mill dataset show that this algorithm can effectively improve the prediction accuracy and generalization ability.
MT-ELM enables multi-task learning using hidden layers to share data characteristics among different tasks.However, MT-ELM ignores correlation differences between tasks and over-fitting problem. Therefore, MT-RELM is proposed. Firstly, RELM is used to solve the over-fitting problem. Secondly, considering correlation differences between tasks, the constraints are added to output weights based on the assumptions that similar task having similar weight, so this paper uses this constraint to indicate relevance level between tasks. Finally, ADMM algorithm are used to solve MT-RELM model parameters. The results based on the synthetic dataset and wet ball mill dataset show that this algorithm can effectively improve the prediction accuracy and generalization ability.
引文
[1] Xue Y,Liao X,Carlin L,et al.Multi-task learning for classification with Dirichlet process priors[J].Mach Learn Res,2007,8:35-63.
[2] Yu K,Schwaighofer A,Schwaighofer A.Learning Gaussian processes from multiple tasks[C]//International Conference on Machine Learning,2005:1012-1019.
[3] Argyriou A,Evgeniou T,Pontil M.Convex multi-task feature learning[J].Machine Learning,2008,73(3):243-272.
[4] Torralba A,Murphy K P,Freeman W T.Sharing visual features for multiclass and multiview object detection[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2007,29(5):854-869.
[5] Bi J,Xiong T,Yu S,et al.An improved multi-task learning approach with applications in medical diagnosis[C]//European Conference on Machine Learning and Knowledge Discovery in Databases,2008:117-132.
[6] Obozinski G,Taskar B,Jordan M I.Joint covariate selection and joint subspace selection for multiple classification problems[J].Statistics and Computing,2010,20(2):231-252.
[7] He X,Mourot G,Maquin D,et al.Multi-task learning with one-class SVM[J].Neurocomputing,2014,133(8):416-426.
[8] Xu S,An X,Qiao X,et al.Multi-task least-squares support vector machines[J].Multimedia Tools&Applications,2014,71(2):699-715.
[9] Huang G B,Zhu Q Y,Siew C K.Extreme learning machine:theory and applications[J].Neurocomputing,2006,70(1/3):489-501.
[10]杨晶晶,李隽,齐志刚,等.基于极限学习机的电力配网物资需求预测[C]//中国电机工程学会年会,2012.
[11] Mao W T,Xu J C,Zhao S J,et al.Research of multi-task learning based on extreme learning machine[J].International Journal of Uncertainty,Fuzziness and KnowledgeBased Systems,2013,21:1340017.
[12]邓万宇,郑庆华,陈琳,等.神经网络极速学习方法研究[J].计算机学报,2010,33(2):279-287.
[13] Chartrand R,Wohlberg B.A nonconvex ADMM algorithm for group sparsity with sparse groups[C]//IEEE International Conference on Acoustics,Speech and Signal Processing,2013:6009-6013.
[14]甘露.极限学习机的研究与应用[D].西安:西安电子科技大学,2014.
[15] Liang L,Cherkassky V.Connection between SVM+and multi-task learning[C]//IEEE International Joint Conference on Neural Networks,2008:2048-2054.
[16] Grigorievskiy A,Miche Y,Ventel?A M,et al.Long-term time series prediction using OP-ELM[J].Neural Networks the Official Journal of the International Neural Network Society,2014,51(C):50.
[17] Lu H,Du B,Liu J,et al.A kernel extreme learning machine algorithm based on improved particle swam optimization[J].Memetic Computing,2017,9(2):121-128.
[18] Zhai J,Wang J,Hu W.Combination of OSELM classifiers with fuzzy integral for large scale classification[J].Journal of Intelligent and FuzzySytems,2015,28(5):2257-2268.
[2] Yu K,Schwaighofer A,Schwaighofer A.Learning Gaussian processes from multiple tasks[C]//International Conference on Machine Learning,2005:1012-1019.
[3] Argyriou A,Evgeniou T,Pontil M.Convex multi-task feature learning[J].Machine Learning,2008,73(3):243-272.
[4] Torralba A,Murphy K P,Freeman W T.Sharing visual features for multiclass and multiview object detection[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2007,29(5):854-869.
[5] Bi J,Xiong T,Yu S,et al.An improved multi-task learning approach with applications in medical diagnosis[C]//European Conference on Machine Learning and Knowledge Discovery in Databases,2008:117-132.
[6] Obozinski G,Taskar B,Jordan M I.Joint covariate selection and joint subspace selection for multiple classification problems[J].Statistics and Computing,2010,20(2):231-252.
[7] He X,Mourot G,Maquin D,et al.Multi-task learning with one-class SVM[J].Neurocomputing,2014,133(8):416-426.
[8] Xu S,An X,Qiao X,et al.Multi-task least-squares support vector machines[J].Multimedia Tools&Applications,2014,71(2):699-715.
[9] Huang G B,Zhu Q Y,Siew C K.Extreme learning machine:theory and applications[J].Neurocomputing,2006,70(1/3):489-501.
[10]杨晶晶,李隽,齐志刚,等.基于极限学习机的电力配网物资需求预测[C]//中国电机工程学会年会,2012.
[11] Mao W T,Xu J C,Zhao S J,et al.Research of multi-task learning based on extreme learning machine[J].International Journal of Uncertainty,Fuzziness and KnowledgeBased Systems,2013,21:1340017.
[12]邓万宇,郑庆华,陈琳,等.神经网络极速学习方法研究[J].计算机学报,2010,33(2):279-287.
[13] Chartrand R,Wohlberg B.A nonconvex ADMM algorithm for group sparsity with sparse groups[C]//IEEE International Conference on Acoustics,Speech and Signal Processing,2013:6009-6013.
[14]甘露.极限学习机的研究与应用[D].西安:西安电子科技大学,2014.
[15] Liang L,Cherkassky V.Connection between SVM+and multi-task learning[C]//IEEE International Joint Conference on Neural Networks,2008:2048-2054.
[16] Grigorievskiy A,Miche Y,Ventel?A M,et al.Long-term time series prediction using OP-ELM[J].Neural Networks the Official Journal of the International Neural Network Society,2014,51(C):50.
[17] Lu H,Du B,Liu J,et al.A kernel extreme learning machine algorithm based on improved particle swam optimization[J].Memetic Computing,2017,9(2):121-128.
[18] Zhai J,Wang J,Hu W.Combination of OSELM classifiers with fuzzy integral for large scale classification[J].Journal of Intelligent and FuzzySytems,2015,28(5):2257-2268.