动态数据流分析的在线超限学习算法综述
|
摘要
动态数据流分析是一个具有广泛应用价值的研究课题,在线学习方法是其中的一种关键技术。在众多在线学习方法中,在线贯序超限学习机(Online Sequential Extreme Learning Machine,OSELM)是一种新颖且实用的在线学习算法,目前已在动态数据流分析中得到了成功应用。首先,介绍了OSELM的理论基础和算法执行过程;然后,以动态数据流分析为应用背景,对各种改进OSELM算法进行了分类综述,包括基于滑动窗口的OSELM、基于遗忘因子的OSELM、基于样本加权的OSELM以及其他方法,重点论述了各类算法的设计思路和实现策略,并对其优缺点进行了比较和分析;最后,探讨了值得进一步研究的问题。
Dynamic data stream analysis has become a research focus for its widespread application prospects,and online learning method is key to solve this problem.Among existing online learning methods,online sequential extreme lear-ning machine(OSELM) is a novel and practical online learning algorithm,and it has been successfully applied in the field of dynamic data stream analysis.Firstly,the theoretical foundation and the execution process of OSELM were reviewed.Then,regarding dynamic data flow analysis as the application background,this paper classified and summarized different kinds of improved OSELM algorithms,including the sliding window based OSELM,forgetting factor based OSELM,sample weighting based OSELM and other methods.This paper focused on the design ideas and implementation strategies of different kinds of algorithms,compared and analyzed the advantages and disadvantages of main algorithms.Finally,the possible works for future research were presented.
Dynamic data stream analysis has become a research focus for its widespread application prospects,and online learning method is key to solve this problem.Among existing online learning methods,online sequential extreme lear-ning machine(OSELM) is a novel and practical online learning algorithm,and it has been successfully applied in the field of dynamic data stream analysis.Firstly,the theoretical foundation and the execution process of OSELM were reviewed.Then,regarding dynamic data flow analysis as the application background,this paper classified and summarized different kinds of improved OSELM algorithms,including the sliding window based OSELM,forgetting factor based OSELM,sample weighting based OSELM and other methods.This paper focused on the design ideas and implementation strategies of different kinds of algorithms,compared and analyzed the advantages and disadvantages of main algorithms.Finally,the possible works for future research were presented.
引文
[1] OLORUNNIMBE M K,VIKTOR H L,PAQUET E.Dynamic adaptation of online ensembles for drifting data streams[J].Journal of Intelligent Information Systems,2018,50(2):291-313.
[2] MEJRI D,LIMAM M,WEIHS C.A new dynamic weighted majority control chart for data streams[J].Soft Computing,2018,22(2):511-522.
[3] MENCAGLI G,TORQUATI M,DANELUTTO M.Elastic- PPQ:A two-level autonomic system for spatial preference query processing over dynamic data streams[J].Future Generation Computer Systems,2018,79(3):862-877.
[4] MOHAMAD S,BOUCHACHIA A,SAYEDMOUCHAWEH M.A Bi-Criteria Active Learning Algorithm for Dynamic Data Streams[J].IEEE Transactions on Neural Networks & Lear-ning Systems,2018,29(1):74-86.
[5] KHAMASSI I,SAYED-MOUCHAWEH M,HAMMAMI M,et al.Discussion and review on evolving data streams and concept drift adapting[J].Evolving Systems,2018,9(1):1-23.
[6] LI C,WEI F,DONG W,et al.Dynamic Structure Embedded Online Multiple-Output Regression for Streaming Data[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2018,doi:10.1109/TPAMI.2018.2794446.
[7] LIANG N Y,HUANG G B,SARATCHANDRAN P,et al.A fast and accurate online sequential learning algorithm for feedforward networks[J].IEEE Transactions on Neural Networks,2006,17(6):1411-23.
[8] HUANG G B,ZHU Q Y,SIEW C K.Extreme learning ma- chine:Theory and applications[J].Neurocomputing,2006,70(1):489-501.
[9] HUYNH H T,WON Y.Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks[J].Pattern Recognition Letters,2011,32(14):1930-1935.
[10] HUANG S,WANG B,QIU J,et al.Parallel ensemble of online sequential extreme learning machine based on MapReduce[J].Neurocomputing,2016,174:352-367.
[11] SCARDAPANE S,COMMINIELLO D,SCARPINITI M,et al.Online Sequential Extreme Learning Machine With Kernels[J].IEEE Transactions on Neural Networks & Learning Systems,2015,26(9):2214-2220.
[12] JIA X,WANG R,LIU J,et al.A semi-supervised online sequential extreme learning machine method[J].Neurocomputing,2016,174:168-178.
[13] WONG P K,XIANG H G,WONG K I,et al.Online extreme learning machine based modeling and optimization for point-by-point engine calibration[J].Neurocomputing,2018,277:187-197.
[14] OUYANG J Q,ZHOU Y,TANG H R.A Meteorological Predication Model Based on Storm and Online Sequential Extreme Learning Machine[J].Journal of Computer Research & Deve-lopment,2017,54(8):1736-1743.(in Chinese)欧阳建权,周勇,唐欢容.基于Storm的在线序列极限学习机的气象预测模型[J].计算机研究与发展,2017,54(8):1736-1743.
[15] ZHOU X,LIU Z,ZHU C.Online Regularized and Kernelized Extreme Learning Machines with Forgetting Mechanism[J].Mathematical Problems in Engineering,2014,2014:1-11.
[16] ZHOU X,WANG C.Cholesky factorization based online regularized and kernelized extreme learning machines with forgetting mechanism[J].Neurocomputing,2016,174:1147-1155.
[17] ZHANG H,ZHANG S,YIN Y.Kernel online sequential ELM algorithm with sliding window subject to time-varying environments[J].Memetic Computing,2016,10(1):43-52.
[18] WANG X,HAN M.Online sequential extreme learning machine with kernels for nonstationary time series prediction[J].Neurocomputing,2014,145:90-97.
[19] ZHANG X,WANG H L.Fixed-memory extreme learning machine and its applications[J].Control and Decision,2012,27(8):1206-1210.(in Chinese)张弦,王宏力.限定记忆极端学习机及其应用[J].控制与决策,2012,27(8):1206-1210.
[20] ZHAO J,WANG Z,PARK D S.Online sequential extreme learning machine with forgetting mechanism[J].Neurocompu-ting,2012,87:79-89.
[21] WANG J,RAN R,ZHOU Y.A Short-Term Photovoltaic Power Prediction Model Based on an FOS-ELM Algorithm[J].Applied Sciences,2017,7(4):423.
[22] XIAO D,LI B,ZHANG S.An online sequential multiple hidden layers extreme learning machine method with forgetting mechanism[J].Chemometrics and Intelligent Laboratory Systems,2018,176:126-133.
[23] MIRZA B,LIN Z.Meta-cognitive online sequential extreme learning machine for imbalanced and concept-drifting data classification[J].Neural Networks,2016,80:79-94.
[24] ZHANG X,WANG H L.Selective forgetting extreme learning machine and its application to time series prediction[J].Acta Physica Sinica,2011,60(8):68-74.(in Chinese)张弦,王宏力.具有选择与遗忘机制的极端学习机在时间序列预测中的应用[J].物理学报,2011,60(8):68-74.
[25] LU J,HUANG J,LU F.Sensor Fault Diagnosis for Aero Engine Based on Online Sequential Extreme Learning Machine with Memory Principle[J].Energies,2017,10(1):1-15.
[26] TIAN Z,WANG G,REN Y,et al.An Adaptive Online Sequential Extreme Learning Machine for Short-Term Wind Speed Prediction Based on Improved Artificial Bee Colony Algorithm[J].Neural Network World,2018,28(3):191-212.
[27] GUO W,XU T,TANG K M,et al.Online sequential extreme learning machine with generalized regularization and forgetting mechanism[J].Control & Decision,2017,32(2):247-254.(in Chinese)郭威,徐涛,汤克明,等.具有广义正则化与遗忘机制的在线贯序超限学习机[J].控制与决策,2017,32(2):247-254.
[28] DU Z L,LI X M,ZHENG Z G,et al.Extreme learning machine based on regularization and forgetting factor and its application in fault prediction[J].Chinese Journal of Scientific Instrument,2015,36(7):1546-1553.(in Chinese)杜占龙,李小民,郑宗贵,等.基于正则化与遗忘因子的极限学习机及其在故障预测中的应用[J].仪器仪表学报,2015,36(7):1546-1553.
[29] WU Z,TANG H,HE S,et al.Fast dynamic hysteresis modeling using a regularized online sequential extreme learning machine with forgetting property[J].The International Journal of Advanced Manufacturing Technology,2018,94:3473-3484.
[30] ZHANG H,ZHANG S,YIN Y.Online sequential ELM algo- rithm with forgetting factor for real applications[J].Neurocomputing,2017,261:144-152.
[31] LIU D,WU Y,JIANG H.FP-ELM:An online sequential lear- ning algorithm for dealing with concept drift[J].Neurocompu-ting,2016,207:322-334.
[32] LI Y,ZHANG S,YIN Y,et al.A Novel Online Sequential Extreme Learning Machine for Gas Utilization Ratio Prediction in Blast Furnaces[J].Sensors,2017,17(8):1847.
[33] LU J,HUANG J,LU F.Time Series Prediction Based on Adaptive Weight Online Sequential Extreme Learning Machine[J].Applied Sciences,2017,7(3):217.
[34] GUO W,XU T,YU J J,et al.Online Sequential Extreme Lear- ning Machine Based on M-estimator and Variable Forgetting Factor[J].Journal of Electronics & Information Technology,2018,40(6):1360-1367.(in Chinese)郭威,徐涛,于建江,等.基于M-estimator与可变遗忘因子的在线贯序超限学习机[J].电子与信息学报,2018,40(6):1360-1367.
[35] SOARES S G,ARA JO R.An adaptive ensemble of online Extreme Learning Machines with variable forgetting factor for dynamic system prediction[J].Neurocomputing,2016,171:693-707.
[36] LIMJS,LEES K,PANGH S,et al.Low complexity adaptive forgetting factor for online sequential extremelearning machine (OS-ELM) for application to nonstationary systemestimations[J].Neural Computing & Applications,2013,22(3-4):569-576.
[37] LIU J.Adaptive forgetting factor OS-ELM and bootstrap for time series prediction[J].International Journal of Modeling Simulation & Scientific Computing,2017,8(3):1-19.
[38] GUO W,XU T,TANG K,et al.Online Sequential Extreme Learning Machine with Generalized Regularization and Adaptive Forgetting Factor for Time-Varying System Prediction[J].Mathematical Problems in Engineering,2018,2018:1-22.
[39] ZOU Q Y,WANG X J,ZHOU C J,et al.The memory degradation based online sequential extreme learning machine[J].Neurocomputing,2018,275:2864-2879.
[40] GU Y,LIU J,CHEN Y,et al.TOSELM:Timeliness Online Sequential Extreme Learning Machine[J].Neurocomputing,2014,128:119-127.
[41] LUO X,YANG X,JIANG C,et al.Timeliness online regularized extreme learning machine[J].International Journal of Machine Learning and Cybernetics,2016,9(3):465-476.
[42] MAO W,WANG J,WANG L,et al.Online Sequential Prediction for Nonstationary Time Series with New Weight-Setting Strategy Using Extreme Learning Machine[J].Mathematical Problems in Engineering,2015,2015:1-13.
[43] LU X,ZHOU C,HUANG M,et al.Regularized online sequential extreme learning machine with adaptive regulation factor for time-varying nonlinear system[J].Neurocomputing,2016,174:617-626.
[44] SAHANI M,DASH P K.Variational mode decomposition and weighted online sequential extreme learning machine for power quality event patterns recognition[J].Neurocomputing,2018,310:10-27.
[45] YIN J C.A variable-structure online sequential extreme learning machine for time-varying system prediction[J].Neurocompu-ting,2017,261:115-125.
[46] XU S,WANG J.Dynamic extreme learning machine for data stream classification[J].Neurocomputing,2017,238:433-449.
[47] YE Y,SQUARTINI S,PIAZZA F.Online sequential extreme learning machine in nonstationary environments[J].Neurocomputing,2013,116:94-101.
[48] YIN J C,ZOU Z J,XU F,et al.Online ship roll motion prediction based on grey sequential extreme learning machine[J].Neurocomputing,2014,129:168-174.
[49] MIRZA B,LIN Z,LIU N.Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift[J].Neurocomputing,2015,149:316-329.
[50] GHIMIRE D,LEE J.Online sequential extreme learning ma- chine-based co-training for dynamic moving cast shadow detection[J].Multimedia Tools and Applications,2015,75(18):11181-11197.
[51] CAI Y,LAMARE R C D,ZHAO M,et al.Low-Com- plexity Variable Forgetting Factor Mechanism for Blind Adaptive Constrained Constant Modulus Algorithms[J].IEEE Transactions on Signal Processing,2012,60(8):3988-4002.
[52] GUO W,XU T,TANG K.M-estimator-based online sequential extreme learning machine for predicting chaotic time series with outliers[J].Neural Computing & Applications,2017,28(12):4093-4110.
[2] MEJRI D,LIMAM M,WEIHS C.A new dynamic weighted majority control chart for data streams[J].Soft Computing,2018,22(2):511-522.
[3] MENCAGLI G,TORQUATI M,DANELUTTO M.Elastic- PPQ:A two-level autonomic system for spatial preference query processing over dynamic data streams[J].Future Generation Computer Systems,2018,79(3):862-877.
[4] MOHAMAD S,BOUCHACHIA A,SAYEDMOUCHAWEH M.A Bi-Criteria Active Learning Algorithm for Dynamic Data Streams[J].IEEE Transactions on Neural Networks & Lear-ning Systems,2018,29(1):74-86.
[5] KHAMASSI I,SAYED-MOUCHAWEH M,HAMMAMI M,et al.Discussion and review on evolving data streams and concept drift adapting[J].Evolving Systems,2018,9(1):1-23.
[6] LI C,WEI F,DONG W,et al.Dynamic Structure Embedded Online Multiple-Output Regression for Streaming Data[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2018,doi:10.1109/TPAMI.2018.2794446.
[7] LIANG N Y,HUANG G B,SARATCHANDRAN P,et al.A fast and accurate online sequential learning algorithm for feedforward networks[J].IEEE Transactions on Neural Networks,2006,17(6):1411-23.
[8] HUANG G B,ZHU Q Y,SIEW C K.Extreme learning ma- chine:Theory and applications[J].Neurocomputing,2006,70(1):489-501.
[9] HUYNH H T,WON Y.Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks[J].Pattern Recognition Letters,2011,32(14):1930-1935.
[10] HUANG S,WANG B,QIU J,et al.Parallel ensemble of online sequential extreme learning machine based on MapReduce[J].Neurocomputing,2016,174:352-367.
[11] SCARDAPANE S,COMMINIELLO D,SCARPINITI M,et al.Online Sequential Extreme Learning Machine With Kernels[J].IEEE Transactions on Neural Networks & Learning Systems,2015,26(9):2214-2220.
[12] JIA X,WANG R,LIU J,et al.A semi-supervised online sequential extreme learning machine method[J].Neurocomputing,2016,174:168-178.
[13] WONG P K,XIANG H G,WONG K I,et al.Online extreme learning machine based modeling and optimization for point-by-point engine calibration[J].Neurocomputing,2018,277:187-197.
[14] OUYANG J Q,ZHOU Y,TANG H R.A Meteorological Predication Model Based on Storm and Online Sequential Extreme Learning Machine[J].Journal of Computer Research & Deve-lopment,2017,54(8):1736-1743.(in Chinese)欧阳建权,周勇,唐欢容.基于Storm的在线序列极限学习机的气象预测模型[J].计算机研究与发展,2017,54(8):1736-1743.
[15] ZHOU X,LIU Z,ZHU C.Online Regularized and Kernelized Extreme Learning Machines with Forgetting Mechanism[J].Mathematical Problems in Engineering,2014,2014:1-11.
[16] ZHOU X,WANG C.Cholesky factorization based online regularized and kernelized extreme learning machines with forgetting mechanism[J].Neurocomputing,2016,174:1147-1155.
[17] ZHANG H,ZHANG S,YIN Y.Kernel online sequential ELM algorithm with sliding window subject to time-varying environments[J].Memetic Computing,2016,10(1):43-52.
[18] WANG X,HAN M.Online sequential extreme learning machine with kernels for nonstationary time series prediction[J].Neurocomputing,2014,145:90-97.
[19] ZHANG X,WANG H L.Fixed-memory extreme learning machine and its applications[J].Control and Decision,2012,27(8):1206-1210.(in Chinese)张弦,王宏力.限定记忆极端学习机及其应用[J].控制与决策,2012,27(8):1206-1210.
[20] ZHAO J,WANG Z,PARK D S.Online sequential extreme learning machine with forgetting mechanism[J].Neurocompu-ting,2012,87:79-89.
[21] WANG J,RAN R,ZHOU Y.A Short-Term Photovoltaic Power Prediction Model Based on an FOS-ELM Algorithm[J].Applied Sciences,2017,7(4):423.
[22] XIAO D,LI B,ZHANG S.An online sequential multiple hidden layers extreme learning machine method with forgetting mechanism[J].Chemometrics and Intelligent Laboratory Systems,2018,176:126-133.
[23] MIRZA B,LIN Z.Meta-cognitive online sequential extreme learning machine for imbalanced and concept-drifting data classification[J].Neural Networks,2016,80:79-94.
[24] ZHANG X,WANG H L.Selective forgetting extreme learning machine and its application to time series prediction[J].Acta Physica Sinica,2011,60(8):68-74.(in Chinese)张弦,王宏力.具有选择与遗忘机制的极端学习机在时间序列预测中的应用[J].物理学报,2011,60(8):68-74.
[25] LU J,HUANG J,LU F.Sensor Fault Diagnosis for Aero Engine Based on Online Sequential Extreme Learning Machine with Memory Principle[J].Energies,2017,10(1):1-15.
[26] TIAN Z,WANG G,REN Y,et al.An Adaptive Online Sequential Extreme Learning Machine for Short-Term Wind Speed Prediction Based on Improved Artificial Bee Colony Algorithm[J].Neural Network World,2018,28(3):191-212.
[27] GUO W,XU T,TANG K M,et al.Online sequential extreme learning machine with generalized regularization and forgetting mechanism[J].Control & Decision,2017,32(2):247-254.(in Chinese)郭威,徐涛,汤克明,等.具有广义正则化与遗忘机制的在线贯序超限学习机[J].控制与决策,2017,32(2):247-254.
[28] DU Z L,LI X M,ZHENG Z G,et al.Extreme learning machine based on regularization and forgetting factor and its application in fault prediction[J].Chinese Journal of Scientific Instrument,2015,36(7):1546-1553.(in Chinese)杜占龙,李小民,郑宗贵,等.基于正则化与遗忘因子的极限学习机及其在故障预测中的应用[J].仪器仪表学报,2015,36(7):1546-1553.
[29] WU Z,TANG H,HE S,et al.Fast dynamic hysteresis modeling using a regularized online sequential extreme learning machine with forgetting property[J].The International Journal of Advanced Manufacturing Technology,2018,94:3473-3484.
[30] ZHANG H,ZHANG S,YIN Y.Online sequential ELM algo- rithm with forgetting factor for real applications[J].Neurocomputing,2017,261:144-152.
[31] LIU D,WU Y,JIANG H.FP-ELM:An online sequential lear- ning algorithm for dealing with concept drift[J].Neurocompu-ting,2016,207:322-334.
[32] LI Y,ZHANG S,YIN Y,et al.A Novel Online Sequential Extreme Learning Machine for Gas Utilization Ratio Prediction in Blast Furnaces[J].Sensors,2017,17(8):1847.
[33] LU J,HUANG J,LU F.Time Series Prediction Based on Adaptive Weight Online Sequential Extreme Learning Machine[J].Applied Sciences,2017,7(3):217.
[34] GUO W,XU T,YU J J,et al.Online Sequential Extreme Lear- ning Machine Based on M-estimator and Variable Forgetting Factor[J].Journal of Electronics & Information Technology,2018,40(6):1360-1367.(in Chinese)郭威,徐涛,于建江,等.基于M-estimator与可变遗忘因子的在线贯序超限学习机[J].电子与信息学报,2018,40(6):1360-1367.
[35] SOARES S G,ARA JO R.An adaptive ensemble of online Extreme Learning Machines with variable forgetting factor for dynamic system prediction[J].Neurocomputing,2016,171:693-707.
[36] LIMJS,LEES K,PANGH S,et al.Low complexity adaptive forgetting factor for online sequential extremelearning machine (OS-ELM) for application to nonstationary systemestimations[J].Neural Computing & Applications,2013,22(3-4):569-576.
[37] LIU J.Adaptive forgetting factor OS-ELM and bootstrap for time series prediction[J].International Journal of Modeling Simulation & Scientific Computing,2017,8(3):1-19.
[38] GUO W,XU T,TANG K,et al.Online Sequential Extreme Learning Machine with Generalized Regularization and Adaptive Forgetting Factor for Time-Varying System Prediction[J].Mathematical Problems in Engineering,2018,2018:1-22.
[39] ZOU Q Y,WANG X J,ZHOU C J,et al.The memory degradation based online sequential extreme learning machine[J].Neurocomputing,2018,275:2864-2879.
[40] GU Y,LIU J,CHEN Y,et al.TOSELM:Timeliness Online Sequential Extreme Learning Machine[J].Neurocomputing,2014,128:119-127.
[41] LUO X,YANG X,JIANG C,et al.Timeliness online regularized extreme learning machine[J].International Journal of Machine Learning and Cybernetics,2016,9(3):465-476.
[42] MAO W,WANG J,WANG L,et al.Online Sequential Prediction for Nonstationary Time Series with New Weight-Setting Strategy Using Extreme Learning Machine[J].Mathematical Problems in Engineering,2015,2015:1-13.
[43] LU X,ZHOU C,HUANG M,et al.Regularized online sequential extreme learning machine with adaptive regulation factor for time-varying nonlinear system[J].Neurocomputing,2016,174:617-626.
[44] SAHANI M,DASH P K.Variational mode decomposition and weighted online sequential extreme learning machine for power quality event patterns recognition[J].Neurocomputing,2018,310:10-27.
[45] YIN J C.A variable-structure online sequential extreme learning machine for time-varying system prediction[J].Neurocompu-ting,2017,261:115-125.
[46] XU S,WANG J.Dynamic extreme learning machine for data stream classification[J].Neurocomputing,2017,238:433-449.
[47] YE Y,SQUARTINI S,PIAZZA F.Online sequential extreme learning machine in nonstationary environments[J].Neurocomputing,2013,116:94-101.
[48] YIN J C,ZOU Z J,XU F,et al.Online ship roll motion prediction based on grey sequential extreme learning machine[J].Neurocomputing,2014,129:168-174.
[49] MIRZA B,LIN Z,LIU N.Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift[J].Neurocomputing,2015,149:316-329.
[50] GHIMIRE D,LEE J.Online sequential extreme learning ma- chine-based co-training for dynamic moving cast shadow detection[J].Multimedia Tools and Applications,2015,75(18):11181-11197.
[51] CAI Y,LAMARE R C D,ZHAO M,et al.Low-Com- plexity Variable Forgetting Factor Mechanism for Blind Adaptive Constrained Constant Modulus Algorithms[J].IEEE Transactions on Signal Processing,2012,60(8):3988-4002.
[52] GUO W,XU T,TANG K.M-estimator-based online sequential extreme learning machine for predicting chaotic time series with outliers[J].Neural Computing & Applications,2017,28(12):4093-4110.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |