基于GDA的置信规则库参数训练的集成学习方法
|
摘要
目前对置信规则库(belief rule base,BRB)的研究主要针对单个BRB系统,然而单个BRB系统的推理性能不仅受参数取值的影响,而且当训练集分布不均衡或数据量较少时,容易导致参数训练不全面,从而使得推理结果所提供的决策信息存在局部性。通过引入Bagging算法和Ada Boost算法,分别与BRB相结合提出了基于梯度下降法(gradient descent algorithm,GDA)的置信规则库系统的集成学习方法,并分别应用于输油管道检漏、多峰函数的置信规则库训练,将多个BRB子系统集成,提高系统的推理性能。在实验中,以收敛精度和曲线拟合效果作为衡量指标来分析集成系统的性能,并将集成系统与其他单个BRB系统进行比较,实验结果表明BRB集成学习方法合理有效。
Current research on belief rule base(BRB) focuses on single BRB system, however, the reasoning performance of single BRB system is influenced by the values of parameters. And the uneven distribution or small amount of training data can lead to the incompleteness of training parameters, which makes the locality of information for decision provided by reasoning results. To solve these problems, this paper proposes BRB-ensemble system base in gradient descent algorithm(GDA) via combining the Bagging and Ada Boost with BRB respectively, and the BRB system is applied to the pipeline leak detection and multimodal function fitting. The performance of BRB system can be improved by the integration of multiple sub-BRB. In the case study, the convergence accuracy and fitting effect are used to analyze the performance of BRB-ensemble, and the proposed approach is compared with other single BRB system.The experimental results show that the BRB-ensemble method is reasonable and effective.
Current research on belief rule base(BRB) focuses on single BRB system, however, the reasoning performance of single BRB system is influenced by the values of parameters. And the uneven distribution or small amount of training data can lead to the incompleteness of training parameters, which makes the locality of information for decision provided by reasoning results. To solve these problems, this paper proposes BRB-ensemble system base in gradient descent algorithm(GDA) via combining the Bagging and Ada Boost with BRB respectively, and the BRB system is applied to the pipeline leak detection and multimodal function fitting. The performance of BRB system can be improved by the integration of multiple sub-BRB. In the case study, the convergence accuracy and fitting effect are used to analyze the performance of BRB-ensemble, and the proposed approach is compared with other single BRB system.The experimental results show that the BRB-ensemble method is reasonable and effective.
引文
[1]Dempster A P.A generalization of Bayesian inference[J].Journal of the Royal Statistical Society:Series B(Methodological),1968,30(2):205-247.
[2]Glenn S.A mathematical theory of evidence[M].Princeton,USA:Princeton University Press,1976.
[3]Zadeh L Z.Fuzzy sets[J].Information and Control,1965,8(3):338-353.
[4]McC ulloch W S,Pitts W.A logical calculus of the ideas immanent in nervous activity[J].The Bulletin of Mathematical Biophysics,1943,5(4):115-133.
[5]Miao Yangzi,Fang Jian,Ma Xiaoping.D-S evidence theory of fusion technology and application[M].Beijing:Publishing House of Electronics Industry,2013.
[6]Yang Jianbo,Liu Jun,Wang Jin,et al.Belief rule-base inference methodology using the evidential reasoning approachRIMER[J].IEEE Transactions on Systems,Man,and Cybernetics:Part A Systems and Humans,2006,36(2):266-285.
[7]Hwang C,Yoon K.Methods for multiple attribute decision making[M].Heidelberg,Berlin:Springer-Verlag,1981:58-191.
[8]Sun R.Robust reasoning:integration rule-based and similaritybased reasoning[J].Artificial Intelligence,1995,75(2):241-295.
[9]Jiang Jiang,Li Xuan,Zhou Zhijie,et al.Weapon system capability assessment under uncertainty based on the evidential reasoning approach[J].Expert Systems with Applications,2011,38(11):13773-13784.
[10]Yang Jianbo,Liu Jun,Xu Dongling,et al.optimization models for training belief-rule-based systems[J].IEEE Transactions on Systems,Man and Cybernetics:Part A Systems and Humans,2007,37(4):569-585.
[11]Zhou Zhijie,Hu Changhua,Yang Jianbo,et al.Online updating belief-rule-based system for pipeline leak detection under expert intervention[J].Expert Systems with Applications,2009,36(4):7700-7709.
[12]Liu Jun,Martinez L,Calzada A,et al.A novel belief rule base representation,generation and its inference methodology[J].Knowledge-Based Systems,2013,53:129-141.
[13]Su Qun,Yang Longhao,Fu Yanggeng,et al.Parameter training approach based on variable particle swarm optimization for belief rule base[J].Journal of Computer Applications,2014,34(8):2161-2165.
[14]Wang Hanjie,Yang Longhao,Fu Yanggeng,et al.Differential evolutionary algorithm for parameter training of belief rule base under expert intervention[J].Computer Science,2015,42(5):88-93.
[15]Zhou Zhijie,Yang Jianbo,Hu Changhua,et al.Belief rule base expert system and complex system modeling[M].Beijing:Science Press,2011:1-119.
[16]Breiman L.Bagging predictors[J].Machine Learning,1996,24(2):123-140.
[17]Freund Y,Schapire R E.A decision-theoretic generalization of on-line learning and an application to boosting[J].Journal of Computer and System Sciences,1997,55(1):119-139.
[18]Freund Y,Schapire R E.Experiments with a new boosting algorithm[C]//Proceedings of the 13th International Conference on Machine Learning,Bari,Italy,Jul 3-6,1996,San Francisco,USA:Morgan Kaufmann Publishers Inc,1996:148-156.
[19]Wu Weikun,Yang Longhao,Fu Yanggeng,et al.Parameter training approach for belief rule base using the accelerating of gradient algorithm[J].Journal of Frontiers of Computer Science and Technology,2014,8(8):989-1001.
[20]Yang Jianbo.Rule and utility based evidential reasoning approach for multi-attribute decision analysis under uncertainties[J].European Journal of Operational Research,2001,131(1):31-61.
[21]Wang Yingming,Yang Jianbo,Xu Dongling,et al.The evidential reasoning approach for multiple attribute decision analysis using interval belief degrees[J].European Journal of Operational Research,2006,175(1):35-66.
[22]Ma Changfeng.Optimization method and Matlab programming[M].Beijing:Science Press,2010:47-189.
[23]Efron B,Tibshirani R J.An introduction to the bootstrap[M].Boca Raton,USA:CRC Press,1994.
[24]Wu Xindong,Kumar V,Quinlan J R,et al.Top 10 algorithms in data mining[J].Knowledge and Information Systems,2008,14(1):1-37.
[25]Drucker H.Improving regressors using boosting techniques[C]//Proceedings of the 14th International Conference on Machine Learning,Nashville,USA,Jul 8-12,1997.San Francisco,USA:Morgan Kaufmann Publishers Inc,1997:107-115.
[26]Hao Liren.SPSS practical statistics[M].Beijing:China Water&Power Press,2003.
[27]Chen Yuwang,Yang Jianbo,Xu Dongling,et al.On the inference and approximation properties of belief rule based systems[J].Information Sciences,2013,234:121-135.
[28]Chang Rui,Wang Hongwei,Yang Jianbo.An algorithm for training parameters in belief rule-bases based on the gradient and dichotomy methods[J].Systems Engineering,2007,6(6):287-291.
[5]缪燕子,方健,马小平,等.D-S证据理论融合技术及其应用[M].北京:电子工业出版社,2013.
[13]苏群,杨隆浩,傅仰耿,等.基于变速粒子群优化的置信规则库参数训练方法[J].计算机应用,2014,34(8):2161-2165.
[14]王韩杰,杨隆浩,傅仰耿,等.专家干预下置信规则库参数训练的差分进化算法[J].计算机科学,2015,42(5):88-93.
[15]周志杰,杨剑波,胡昌华,等.置信规则库专家系统与复杂系统建模[M].北京:科学出版社,2011:1-119.
[19]吴伟昆,杨隆浩,傅仰耿,等.基于加速梯度求法的置信规则库参数训练方法[J].计算机科学与探索,2014,8(8):989-1001.
[22]马昌凤.最优化方法及其Matlab程序设计[M].北京:科学出版社,2010:47-189.
[26]郝黎仁.SPSS实用统计分析[M].北京:中国水利水电出版社,2003.
[2]Glenn S.A mathematical theory of evidence[M].Princeton,USA:Princeton University Press,1976.
[3]Zadeh L Z.Fuzzy sets[J].Information and Control,1965,8(3):338-353.
[4]McC ulloch W S,Pitts W.A logical calculus of the ideas immanent in nervous activity[J].The Bulletin of Mathematical Biophysics,1943,5(4):115-133.
[5]Miao Yangzi,Fang Jian,Ma Xiaoping.D-S evidence theory of fusion technology and application[M].Beijing:Publishing House of Electronics Industry,2013.
[6]Yang Jianbo,Liu Jun,Wang Jin,et al.Belief rule-base inference methodology using the evidential reasoning approachRIMER[J].IEEE Transactions on Systems,Man,and Cybernetics:Part A Systems and Humans,2006,36(2):266-285.
[7]Hwang C,Yoon K.Methods for multiple attribute decision making[M].Heidelberg,Berlin:Springer-Verlag,1981:58-191.
[8]Sun R.Robust reasoning:integration rule-based and similaritybased reasoning[J].Artificial Intelligence,1995,75(2):241-295.
[9]Jiang Jiang,Li Xuan,Zhou Zhijie,et al.Weapon system capability assessment under uncertainty based on the evidential reasoning approach[J].Expert Systems with Applications,2011,38(11):13773-13784.
[10]Yang Jianbo,Liu Jun,Xu Dongling,et al.optimization models for training belief-rule-based systems[J].IEEE Transactions on Systems,Man and Cybernetics:Part A Systems and Humans,2007,37(4):569-585.
[11]Zhou Zhijie,Hu Changhua,Yang Jianbo,et al.Online updating belief-rule-based system for pipeline leak detection under expert intervention[J].Expert Systems with Applications,2009,36(4):7700-7709.
[12]Liu Jun,Martinez L,Calzada A,et al.A novel belief rule base representation,generation and its inference methodology[J].Knowledge-Based Systems,2013,53:129-141.
[13]Su Qun,Yang Longhao,Fu Yanggeng,et al.Parameter training approach based on variable particle swarm optimization for belief rule base[J].Journal of Computer Applications,2014,34(8):2161-2165.
[14]Wang Hanjie,Yang Longhao,Fu Yanggeng,et al.Differential evolutionary algorithm for parameter training of belief rule base under expert intervention[J].Computer Science,2015,42(5):88-93.
[15]Zhou Zhijie,Yang Jianbo,Hu Changhua,et al.Belief rule base expert system and complex system modeling[M].Beijing:Science Press,2011:1-119.
[16]Breiman L.Bagging predictors[J].Machine Learning,1996,24(2):123-140.
[17]Freund Y,Schapire R E.A decision-theoretic generalization of on-line learning and an application to boosting[J].Journal of Computer and System Sciences,1997,55(1):119-139.
[18]Freund Y,Schapire R E.Experiments with a new boosting algorithm[C]//Proceedings of the 13th International Conference on Machine Learning,Bari,Italy,Jul 3-6,1996,San Francisco,USA:Morgan Kaufmann Publishers Inc,1996:148-156.
[19]Wu Weikun,Yang Longhao,Fu Yanggeng,et al.Parameter training approach for belief rule base using the accelerating of gradient algorithm[J].Journal of Frontiers of Computer Science and Technology,2014,8(8):989-1001.
[20]Yang Jianbo.Rule and utility based evidential reasoning approach for multi-attribute decision analysis under uncertainties[J].European Journal of Operational Research,2001,131(1):31-61.
[21]Wang Yingming,Yang Jianbo,Xu Dongling,et al.The evidential reasoning approach for multiple attribute decision analysis using interval belief degrees[J].European Journal of Operational Research,2006,175(1):35-66.
[22]Ma Changfeng.Optimization method and Matlab programming[M].Beijing:Science Press,2010:47-189.
[23]Efron B,Tibshirani R J.An introduction to the bootstrap[M].Boca Raton,USA:CRC Press,1994.
[24]Wu Xindong,Kumar V,Quinlan J R,et al.Top 10 algorithms in data mining[J].Knowledge and Information Systems,2008,14(1):1-37.
[25]Drucker H.Improving regressors using boosting techniques[C]//Proceedings of the 14th International Conference on Machine Learning,Nashville,USA,Jul 8-12,1997.San Francisco,USA:Morgan Kaufmann Publishers Inc,1997:107-115.
[26]Hao Liren.SPSS practical statistics[M].Beijing:China Water&Power Press,2003.
[27]Chen Yuwang,Yang Jianbo,Xu Dongling,et al.On the inference and approximation properties of belief rule based systems[J].Information Sciences,2013,234:121-135.
[28]Chang Rui,Wang Hongwei,Yang Jianbo.An algorithm for training parameters in belief rule-bases based on the gradient and dichotomy methods[J].Systems Engineering,2007,6(6):287-291.
[5]缪燕子,方健,马小平,等.D-S证据理论融合技术及其应用[M].北京:电子工业出版社,2013.
[13]苏群,杨隆浩,傅仰耿,等.基于变速粒子群优化的置信规则库参数训练方法[J].计算机应用,2014,34(8):2161-2165.
[14]王韩杰,杨隆浩,傅仰耿,等.专家干预下置信规则库参数训练的差分进化算法[J].计算机科学,2015,42(5):88-93.
[15]周志杰,杨剑波,胡昌华,等.置信规则库专家系统与复杂系统建模[M].北京:科学出版社,2011:1-119.
[19]吴伟昆,杨隆浩,傅仰耿,等.基于加速梯度求法的置信规则库参数训练方法[J].计算机科学与探索,2014,8(8):989-1001.
[22]马昌凤.最优化方法及其Matlab程序设计[M].北京:科学出版社,2010:47-189.
[26]郝黎仁.SPSS实用统计分析[M].北京:中国水利水电出版社,2003.