约束条件下BN参数最大熵模型扩展学习算法
|
摘要
在很多智能系统的参数建模时,用户往往面对建模样本稀少的困境。针对在小数据集条件下贝叶斯网络(BN)参数建模的问题,提出了一种约束数据最大熵BN参数学习算法(CDME)。首先利用小数据集估算BN参数,随后把定性的专家经验转换为不等式约束,并利用Bootstrap算法生成满足约束的一组参数候选集,再根据信息最大熵进行加权计算出BN参数。实验结果表明,当数据量充分时,CDME参数学习算法与经典的MLE算法的学习精度近似,表明了算法的正确性;在小数据集条件下,利用CDME算法可以对BN进行参数建模,学习精度优于MLE和QMAP算法。CDME算法在实际故障诊断样本数据相对稀缺的条件下,获取了诊断BN模型参数,在此基础上完成的诊断推理结果也印证了算法的有效性,为小数据集条件下的参数建模提供了一条新途径。
While the intelligent systems need parameter modeling,users often face the dilemma of scarce modeling samples.This paper proposed a BN parameter learning method-constrained data maximum entropy(CDME) algorithm for the modeling of BN parameters under the small data sets. In the case of estimating BN parameters by using small data sets,it transformed the qualitative expert knowledge into inequality constraints for the sake of generating candidate parameter sets by Bootstrap algorithm. Then it estimated the BN parameters in the light of the maximum entropy principle. The experimental results show that CDME algorithm learning effects are similar to the classical MLE algorithm when the modeling data size is sufficient. However,when the data size is limited,the parameters of BN can be modeled by using the CDME,and the learnt accuracy is superior to MLE or QMAP algorithm. It also applied CDME to a real fault diagnosis while the data set was relatively scarce. The results of the diagnosis reasoning demonstrate that the presented parameter learning approach is effective. The CDME parameter learning algorithm provides a new modeling way for BN parameter under the small data sets.
While the intelligent systems need parameter modeling,users often face the dilemma of scarce modeling samples.This paper proposed a BN parameter learning method-constrained data maximum entropy(CDME) algorithm for the modeling of BN parameters under the small data sets. In the case of estimating BN parameters by using small data sets,it transformed the qualitative expert knowledge into inequality constraints for the sake of generating candidate parameter sets by Bootstrap algorithm. Then it estimated the BN parameters in the light of the maximum entropy principle. The experimental results show that CDME algorithm learning effects are similar to the classical MLE algorithm when the modeling data size is sufficient. However,when the data size is limited,the parameters of BN can be modeled by using the CDME,and the learnt accuracy is superior to MLE or QMAP algorithm. It also applied CDME to a real fault diagnosis while the data set was relatively scarce. The results of the diagnosis reasoning demonstrate that the presented parameter learning approach is effective. The CDME parameter learning algorithm provides a new modeling way for BN parameter under the small data sets.
引文
[1] Dojer N. Learning Bayesian networks from datasets joining continuous and discrete variables[J]. International Journal of Approximate Reasoning,2016,78(11):116-124.
[2] Liao Wenhui,Ji Qiang. Learning Bayesian network parameters under incomplete data with domain knowledge[J]. Pattern Recognition,2009,42(11):3046-3056.
[3]陈为,朱标,张宏鑫. BN-Mapping:基于贝叶斯网络的地理空间数据可视分析[J].计算机学报,2016,39(7):1281-1293.(Chen Wei,Zhu Biao,Zhang Hongxin. BN-Mapping:visual analysis of geospatial data with Bayesian network[J]. Chinese Journal of Computers,2016,39(7):1281-1293.)
[4]李硕豪,张军.贝叶斯网络结构学习综述[J].计算机应用研究,2015,32(3):641-646.(Li Shuohao,Zhang Jun. Review of Bayesian networks structure learning[J]. Application Research of Computers,2015,32(3):641-646.)
[5]陈静,蒋正凯,付敬奇.基于Netica的自学习贝叶斯网络的构建[J].电子测量与仪器学报,2016,30(11):1687-1693.(Chen Jing,Jiang Zhengkai,Fu Jingqi. Construction of self-learning Bayesian network based on Netica[J]. Journal of Electronic Measurement and Instrumentation,2016,30(11):1687-1693.)
[6]肖蒙,张友鹏.小数据集条件下的多态系统贝叶斯网络参数学习[J].计算机科学,2015,42(4):253-257.(Xiao Meng,Zhang Youpeng. Parameters learning of Bayesian networks for multistate system with small sample[J]. Computer Science,2015,42(4):253-257.)
[7]李子达,廖士中.小样本贝叶斯网络参数学习方法[J].计算机工程,2016,42(8):153-159,165.(Li Zida,Liao Shizhong.Bayesian network parameter learning method on small samples[J].Computer Engineering,2016,42(8):153-159,165.)
[8]郭志高,高晓光,邸若海.小数据集条件下基于双重约束的BN参数学习[J].自动化学报,2014,40(7):1509-1516.(Guo Zhigao,Gao Xiaoguang,Di Ruohai. Learning Bayesian network parameters under dual constraints from small data set[J]. Acta Automatica Sinica,2014,40(7):1509-1516.)
[9] Altendorf E E,Restificar A C,Dietterich T G. Learning from sparse data by exploiting monotonicity constraints[C]//Proc of the 21st Conference on Uncertainty in Artificial Intelligence. Piscataway,NJ:IEEE Press,2005:18-26.
[10]Feelders A,Gaag L C V D. Learning Bayesian networks parameters under order constraints[J]. International Journal of Approximate Reasoning,2006,42(1-2):37-53.
[11] Isozaki T,Kato N,Ueno M.“Data temperature”in minimum free energies for parameter learning of Bayesian networks[J]. International Journal on Artificial Intelligence Tools,2009,18(5):653-671.
[12]Niculescu R,Mitchell M,Rao B R. Bayesian network learning with parameter constraints[J]. Journal of Machine Learning Research,2006,7(1):1357-1383.
[13]Campos C,Tong Yang,Ji Qiang. Constrained maximum likelihood learning of Bayesian networks for facial action recognition[C]//Proc of the 10th European Conference on Computer Vision. Piscataway,NJ:IEEE Press,2008:168-181.
[14] Chang Rui,Shoemaker R,Wang Wei. A novel knowledge-driven systems biology approach for phenotype prediction upon genetic intervention[J]. Trans on Computational Biology and Bioinformatics,2011,1(8):1170-1181.
[15]Guo Zhigao,Gao Xiaoguang,Di Ruohai,et al. Learning Bayesian network parameters from small data set:a spatially maximum a posteriori method[C]//Proc of the 2nd International Workshop on Advanced Methodologies for Bayesian Networks. New York:SpringerVerlag,2015:32-45.
[16] Berger A L,Pietra S,Pietra V. A maximum entropy approach to natural language processing[J]. Computational Linguistics,1996,22(1):38-73.
[17]Kullback S,Leibler R A. On information and sufficiency[J]. Annals of Mathematical Statistics,1951,22(1):79-86.
[18]孙慧玲,胡伟文,刘海涛.小样本情况下参数区间估计的改进方法[J].哈尔滨理工大学学报,2017,22(1):109-113.(Sun Huiling,Hu Weiwen,Liu Haitao. An improvement to interval estimation for small samples[J]. Journal of Harbin University of Science and Technology,2017,22(1):109-113.)
[19] Russell S J,Norvig P. Artificial intelligence:a modern approach[M]. Englewood:Prentice Hall,2010.
[20]郭文强,张宝嵘,彭程,等.基于小波包和BN模型的深沟球轴承故障诊断[J].轴承,2016,59(3):48-52.(Guo Wenqiang,Zhang Baorong,Peng Cheng,et al. Fault diagnosis for deep groove ball bearings based on wavelet packet and BN model[J]. Bearing,2016,59(3):48-52.)
[2] Liao Wenhui,Ji Qiang. Learning Bayesian network parameters under incomplete data with domain knowledge[J]. Pattern Recognition,2009,42(11):3046-3056.
[3]陈为,朱标,张宏鑫. BN-Mapping:基于贝叶斯网络的地理空间数据可视分析[J].计算机学报,2016,39(7):1281-1293.(Chen Wei,Zhu Biao,Zhang Hongxin. BN-Mapping:visual analysis of geospatial data with Bayesian network[J]. Chinese Journal of Computers,2016,39(7):1281-1293.)
[4]李硕豪,张军.贝叶斯网络结构学习综述[J].计算机应用研究,2015,32(3):641-646.(Li Shuohao,Zhang Jun. Review of Bayesian networks structure learning[J]. Application Research of Computers,2015,32(3):641-646.)
[5]陈静,蒋正凯,付敬奇.基于Netica的自学习贝叶斯网络的构建[J].电子测量与仪器学报,2016,30(11):1687-1693.(Chen Jing,Jiang Zhengkai,Fu Jingqi. Construction of self-learning Bayesian network based on Netica[J]. Journal of Electronic Measurement and Instrumentation,2016,30(11):1687-1693.)
[6]肖蒙,张友鹏.小数据集条件下的多态系统贝叶斯网络参数学习[J].计算机科学,2015,42(4):253-257.(Xiao Meng,Zhang Youpeng. Parameters learning of Bayesian networks for multistate system with small sample[J]. Computer Science,2015,42(4):253-257.)
[7]李子达,廖士中.小样本贝叶斯网络参数学习方法[J].计算机工程,2016,42(8):153-159,165.(Li Zida,Liao Shizhong.Bayesian network parameter learning method on small samples[J].Computer Engineering,2016,42(8):153-159,165.)
[8]郭志高,高晓光,邸若海.小数据集条件下基于双重约束的BN参数学习[J].自动化学报,2014,40(7):1509-1516.(Guo Zhigao,Gao Xiaoguang,Di Ruohai. Learning Bayesian network parameters under dual constraints from small data set[J]. Acta Automatica Sinica,2014,40(7):1509-1516.)
[9] Altendorf E E,Restificar A C,Dietterich T G. Learning from sparse data by exploiting monotonicity constraints[C]//Proc of the 21st Conference on Uncertainty in Artificial Intelligence. Piscataway,NJ:IEEE Press,2005:18-26.
[10]Feelders A,Gaag L C V D. Learning Bayesian networks parameters under order constraints[J]. International Journal of Approximate Reasoning,2006,42(1-2):37-53.
[11] Isozaki T,Kato N,Ueno M.“Data temperature”in minimum free energies for parameter learning of Bayesian networks[J]. International Journal on Artificial Intelligence Tools,2009,18(5):653-671.
[12]Niculescu R,Mitchell M,Rao B R. Bayesian network learning with parameter constraints[J]. Journal of Machine Learning Research,2006,7(1):1357-1383.
[13]Campos C,Tong Yang,Ji Qiang. Constrained maximum likelihood learning of Bayesian networks for facial action recognition[C]//Proc of the 10th European Conference on Computer Vision. Piscataway,NJ:IEEE Press,2008:168-181.
[14] Chang Rui,Shoemaker R,Wang Wei. A novel knowledge-driven systems biology approach for phenotype prediction upon genetic intervention[J]. Trans on Computational Biology and Bioinformatics,2011,1(8):1170-1181.
[15]Guo Zhigao,Gao Xiaoguang,Di Ruohai,et al. Learning Bayesian network parameters from small data set:a spatially maximum a posteriori method[C]//Proc of the 2nd International Workshop on Advanced Methodologies for Bayesian Networks. New York:SpringerVerlag,2015:32-45.
[16] Berger A L,Pietra S,Pietra V. A maximum entropy approach to natural language processing[J]. Computational Linguistics,1996,22(1):38-73.
[17]Kullback S,Leibler R A. On information and sufficiency[J]. Annals of Mathematical Statistics,1951,22(1):79-86.
[18]孙慧玲,胡伟文,刘海涛.小样本情况下参数区间估计的改进方法[J].哈尔滨理工大学学报,2017,22(1):109-113.(Sun Huiling,Hu Weiwen,Liu Haitao. An improvement to interval estimation for small samples[J]. Journal of Harbin University of Science and Technology,2017,22(1):109-113.)
[19] Russell S J,Norvig P. Artificial intelligence:a modern approach[M]. Englewood:Prentice Hall,2010.
[20]郭文强,张宝嵘,彭程,等.基于小波包和BN模型的深沟球轴承故障诊断[J].轴承,2016,59(3):48-52.(Guo Wenqiang,Zhang Baorong,Peng Cheng,et al. Fault diagnosis for deep groove ball bearings based on wavelet packet and BN model[J]. Bearing,2016,59(3):48-52.)