基于决策树特征提取的支持向量机在岩性分类中的应用
|
摘要
由于支持向量机属于黑箱模型,因此在进行模型学习时无法直接对特征进行选择,而决策树模型在递归创建的过程中自身具有一定的特征选择能力。针对岩性分类问题,本文将决策树和支持向量机结合,通过决策树的建立,在考虑特征重要性的前提下,利用树节点的高度对特征进行提取,并将具有更高分类能力的特征送入支持向量机进行岩性分类。结果表明:通过决策树的特征提取,减少了支持向量机模型的输入特征,从而有效控制了模型的复杂度,使得模型更加稳定并具有更高的分类精度,测试集精度能够提升10%以上。
Support vector machine is a kind of black box model,and its feature cannot be selected directly when learning model;while decision tree model has the ability of feature selection during the process of recursive creation.For lithology classification,we combined decision tree with support vector machine.In consideration with the importance of the features,we used the tree height to extract the features after the decision tree establishment,and furthermore,we used the features with higher classification ability to fed into the support vector machine.The results show that the feature extraction of decision tree can reduce the input characteristics,so this,in turn,makes the SVM model more stable and accurate through controlling the complexity of the model effectively.The accuracy of test set of the model can be increased by more than 10%.
Support vector machine is a kind of black box model,and its feature cannot be selected directly when learning model;while decision tree model has the ability of feature selection during the process of recursive creation.For lithology classification,we combined decision tree with support vector machine.In consideration with the importance of the features,we used the tree height to extract the features after the decision tree establishment,and furthermore,we used the features with higher classification ability to fed into the support vector machine.The results show that the feature extraction of decision tree can reduce the input characteristics,so this,in turn,makes the SVM model more stable and accurate through controlling the complexity of the model effectively.The accuracy of test set of the model can be increased by more than 10%.
引文
[1]李航.统计学习方法[M].北京:清华大学出版社,2012.Li Hang.Statistical Learning Method[M].Beijing:Tsinghua University Press,2012.
[2]于代国,孙建孟,王焕增,等.测井识别岩性新方法:支持向量机方法[J].大庆石油地质与开发,2005,24(5):93-95.Yu Daiguo,Sun Jianmeng,Wang Huanzeng,et al.ANew Method of Logging Recognition Lithology:Support Vector Machine Method[J].Daqing Petroleum Geology and Development,2005,24(5):93-95.
[3]周继宏,袁瑞.基于支持向量机的复杂碎屑岩储层岩性识别[J].石油天然气学报,2012,34(7):72-75.Zhou Jihong,Yuan Rui.Lithology Identification of Complex Clastic Rock Reservoirs Based on Support Vector Machine[J].Journal of Oil and Gas Technolog,2012,34(7):72-75.
[4]张翔,肖小玲,严良俊,等.基于模糊支持向量机方法的岩性识别[J].石油天然气学报(江汉石油学院学报),2009,31(6):115-118.Zhang Xiang,Xiao Xiaoling,Yan Liangjun,et al.Lithology Identification Based on Fuzzy Support Vector Machine[J].Journal of Oil and Gas Technolog,2009,31(6):115-118.
[5]李洪奇,谭锋奇,许长福,等.基于决策树方法的砾岩油藏岩性识别[J].测井技术,2010,34(1):16-21.Li Hongqi,Tan Fengqi,Xu Changfu,et al.Lithology Identification of Conglomerate Reservoir Based on Decision Tree Method[J].Well Logging Technology,2010,34(1):16-21.
[6]石广仁.支持向量机在裂缝预测及含气性评价应用中的优越性[J].石油勘探与开发,2008,35(5):589-594.Shi Guangren.Superiorities of Support Vector Machine in Fracture Prediction and Gassinesse Valuation[J].Petroleum Exploration and Development,2008,35(5):589-594.
[7]桑吉夫·库尔卡尼,吉尔伯特·哈曼.统计学习理论基础[M].肖忠祥,闫效莺,段沛沛,等译.北京:机械工业出版社,2017.Kulkarni S,Harman G.An Elementary Introduction to Statistical Learining Theory[M].Translated by Xiao Zhongxiang,Yan Xiaoying,Duan Peipei,et al.Beijing:Machinery Industry Press,2017.
[8]王建国,董泽宇,张文兴,等.基于回归树的支持向量机规则提取及应用[J].计算机工程与应用,2017,53(6):236-240.Wang Jianguo,Dong Zeyu,Zhang Wenxing,et al.Rule Extraction of Support Vector Machine Based on Regression Tree and Application[J].Computer Engineering and Applications,2017,53(6):236-240.
[9]Barakat N,Bradley A P.Rule Extraction from Support Vector Machines:A Review[J].Neurocomputing,2010,74(5):178-190.
[10]温小霓,蔡汝骏.分类与回归树及其应用研究[J].统计与决策,2007(23):14-16.Wen Xiaoni,Cai Rujun.Classification and Regression Tree and Its Application Research[J].Statistics and Decision,2007(23):14-16.
[11]谢益辉.基于R软件rpart包的分类与回归树应用[J].统计与信息论坛,2007,22(5):67-70.Xie Yihui.Classification and Regression Tree Application Based on R Software Rpart Package[J].Forum on Statistics and Information,2007,22(5):67-70.
[12]周志华.机器学习[M].北京:清华大学出版社,2016:1-415.Zhou Zhihua.Machine Learning[M].Beijing:Tsinghua University Press,2016:1-415.
[13]范淼,李超.Python机器学习及实践[M].北京:清华大学出版社,2016:1-180.Fan Miao,Li Chao.Python Machine Learning and Practice[M].Beijing:Tsinghua University Press,2016:1-180.
[14]张冰,郭智奇,徐聪,等.基于岩石物理模型的页岩储层裂缝属性及各向异性参数反演[J].吉林大学学报(地球科学版),2018,48(4):1244-1252.Zhang Bing,Guo Zhiqi,Xu Cong,et al.Fracture Properties and Anisotropic Parameters Inversion of Shales Based on Rock Physics Model[J].Journal of Jilin University(Earth Science Edition),2018,48(4):1244-1252.
[15]杨震宇.基于机器学习的分类算法研究[J].科学中国人,2017,2:22-25.Yang Zhenyu.Research on Classification Algorithm Based on Machine Learning[J].Scientific Chinese,2017,2:22-25.
[16]丁世飞,齐丙娟,谭红艳.支持向量机理论与算法研究综述[J].电子科技大学学报,2011,40(1):2-10.Ding Shifei,Qi Bingjuan,Tan Hongyan.An Overview on Theory and Algorithm of Support Vector Machines[J].Journal of University of Electronic Science and Technology of China,2011,40(1):2-10.
[17]冷强奎,李玉鑑.使用SVM和二叉树结构的分片线性分类器[J].中国科技论文,2015,10(2):164-168.Leng Qiangkui,Li Yujian.A Piecewise Linear Classifier Using SVM and two Forked Tree Structure[J].China Sciencepaper,2015,10(2):164-168.
[18]石广仁.支持向量机在多地质因素分析中的应用[J].石油学报,2008,29(2):195-198.Shi Guangren.Application of Support Vector Machine to Multi-Geological-Factor Analysis[J].Acta Petrolei Sinica,2008,29(2):195-198.
[19]Janez D,Dale S.Statistical Comparisons of Classifiers over Multiple DataSets[J].Journal of Machine Learning Research,2006,7(1):1-30.
[20]Zhang M,Zhou Z.A Review on Multi-Label Learning Algorithms[J].IEEE Transactions on Knowledge and Data Engineering,2014,26(8):1819-1837.
[21]Yoonkyung L,Yi L,Grace W.Multicategory Support Vector Machines:Theory and Application to the Classification of Microarray Data and Satellite Radiance Data[J].Journal of the American Statistical Association,2004,99:67-81.
[22]Fan R E,Chang K W,Hsieh C J,et al.LIBLINEAR:A Library for Large Linear Classification[J].Journal of Machine Learning Research,2008,9:1871-1874.
[23]Shwartz S S,Singer Y,Srebro N,et al.Pegasos:Primal Estimated Sub-Gradient Solver for SVM[J].Mathematical Programming,2011,127(1):3-30.
[24]Bouchaffra D,Vitae A,Cheriet M.Machine Learning and Pattern Recognition Models in Change Detection[J].Pattern Recognition,2015,48(3):613-615.
[25]Collins M,Schapire R E,Singer Y.Logistic Regression,Ada Boost and Bregman Distances[J].Machine Learning,2002,48(1):235-285.
[26]Canu S,Smola A.Kernel Methods and the Exponential Family[J].Neurocomputing,2006,69(7):714-720.
[27]Tsochantaridis I,Joachims T,Hofmann T,et al.Large Margin Methods for Structured and Interdependent Output Variables[J].Journal of Machine Learning Research,2005,1:1453-1484.
[28]Chang C C,Lin C J.LIBSVM:A Library for Support Vector Machines[J/OL].ACM Transactions on Intelligent Systems and Technology,2011,2(3).http://dx.doi.org/10.1145/1961189.1961199.
[2]于代国,孙建孟,王焕增,等.测井识别岩性新方法:支持向量机方法[J].大庆石油地质与开发,2005,24(5):93-95.Yu Daiguo,Sun Jianmeng,Wang Huanzeng,et al.ANew Method of Logging Recognition Lithology:Support Vector Machine Method[J].Daqing Petroleum Geology and Development,2005,24(5):93-95.
[3]周继宏,袁瑞.基于支持向量机的复杂碎屑岩储层岩性识别[J].石油天然气学报,2012,34(7):72-75.Zhou Jihong,Yuan Rui.Lithology Identification of Complex Clastic Rock Reservoirs Based on Support Vector Machine[J].Journal of Oil and Gas Technolog,2012,34(7):72-75.
[4]张翔,肖小玲,严良俊,等.基于模糊支持向量机方法的岩性识别[J].石油天然气学报(江汉石油学院学报),2009,31(6):115-118.Zhang Xiang,Xiao Xiaoling,Yan Liangjun,et al.Lithology Identification Based on Fuzzy Support Vector Machine[J].Journal of Oil and Gas Technolog,2009,31(6):115-118.
[5]李洪奇,谭锋奇,许长福,等.基于决策树方法的砾岩油藏岩性识别[J].测井技术,2010,34(1):16-21.Li Hongqi,Tan Fengqi,Xu Changfu,et al.Lithology Identification of Conglomerate Reservoir Based on Decision Tree Method[J].Well Logging Technology,2010,34(1):16-21.
[6]石广仁.支持向量机在裂缝预测及含气性评价应用中的优越性[J].石油勘探与开发,2008,35(5):589-594.Shi Guangren.Superiorities of Support Vector Machine in Fracture Prediction and Gassinesse Valuation[J].Petroleum Exploration and Development,2008,35(5):589-594.
[7]桑吉夫·库尔卡尼,吉尔伯特·哈曼.统计学习理论基础[M].肖忠祥,闫效莺,段沛沛,等译.北京:机械工业出版社,2017.Kulkarni S,Harman G.An Elementary Introduction to Statistical Learining Theory[M].Translated by Xiao Zhongxiang,Yan Xiaoying,Duan Peipei,et al.Beijing:Machinery Industry Press,2017.
[8]王建国,董泽宇,张文兴,等.基于回归树的支持向量机规则提取及应用[J].计算机工程与应用,2017,53(6):236-240.Wang Jianguo,Dong Zeyu,Zhang Wenxing,et al.Rule Extraction of Support Vector Machine Based on Regression Tree and Application[J].Computer Engineering and Applications,2017,53(6):236-240.
[9]Barakat N,Bradley A P.Rule Extraction from Support Vector Machines:A Review[J].Neurocomputing,2010,74(5):178-190.
[10]温小霓,蔡汝骏.分类与回归树及其应用研究[J].统计与决策,2007(23):14-16.Wen Xiaoni,Cai Rujun.Classification and Regression Tree and Its Application Research[J].Statistics and Decision,2007(23):14-16.
[11]谢益辉.基于R软件rpart包的分类与回归树应用[J].统计与信息论坛,2007,22(5):67-70.Xie Yihui.Classification and Regression Tree Application Based on R Software Rpart Package[J].Forum on Statistics and Information,2007,22(5):67-70.
[12]周志华.机器学习[M].北京:清华大学出版社,2016:1-415.Zhou Zhihua.Machine Learning[M].Beijing:Tsinghua University Press,2016:1-415.
[13]范淼,李超.Python机器学习及实践[M].北京:清华大学出版社,2016:1-180.Fan Miao,Li Chao.Python Machine Learning and Practice[M].Beijing:Tsinghua University Press,2016:1-180.
[14]张冰,郭智奇,徐聪,等.基于岩石物理模型的页岩储层裂缝属性及各向异性参数反演[J].吉林大学学报(地球科学版),2018,48(4):1244-1252.Zhang Bing,Guo Zhiqi,Xu Cong,et al.Fracture Properties and Anisotropic Parameters Inversion of Shales Based on Rock Physics Model[J].Journal of Jilin University(Earth Science Edition),2018,48(4):1244-1252.
[15]杨震宇.基于机器学习的分类算法研究[J].科学中国人,2017,2:22-25.Yang Zhenyu.Research on Classification Algorithm Based on Machine Learning[J].Scientific Chinese,2017,2:22-25.
[16]丁世飞,齐丙娟,谭红艳.支持向量机理论与算法研究综述[J].电子科技大学学报,2011,40(1):2-10.Ding Shifei,Qi Bingjuan,Tan Hongyan.An Overview on Theory and Algorithm of Support Vector Machines[J].Journal of University of Electronic Science and Technology of China,2011,40(1):2-10.
[17]冷强奎,李玉鑑.使用SVM和二叉树结构的分片线性分类器[J].中国科技论文,2015,10(2):164-168.Leng Qiangkui,Li Yujian.A Piecewise Linear Classifier Using SVM and two Forked Tree Structure[J].China Sciencepaper,2015,10(2):164-168.
[18]石广仁.支持向量机在多地质因素分析中的应用[J].石油学报,2008,29(2):195-198.Shi Guangren.Application of Support Vector Machine to Multi-Geological-Factor Analysis[J].Acta Petrolei Sinica,2008,29(2):195-198.
[19]Janez D,Dale S.Statistical Comparisons of Classifiers over Multiple DataSets[J].Journal of Machine Learning Research,2006,7(1):1-30.
[20]Zhang M,Zhou Z.A Review on Multi-Label Learning Algorithms[J].IEEE Transactions on Knowledge and Data Engineering,2014,26(8):1819-1837.
[21]Yoonkyung L,Yi L,Grace W.Multicategory Support Vector Machines:Theory and Application to the Classification of Microarray Data and Satellite Radiance Data[J].Journal of the American Statistical Association,2004,99:67-81.
[22]Fan R E,Chang K W,Hsieh C J,et al.LIBLINEAR:A Library for Large Linear Classification[J].Journal of Machine Learning Research,2008,9:1871-1874.
[23]Shwartz S S,Singer Y,Srebro N,et al.Pegasos:Primal Estimated Sub-Gradient Solver for SVM[J].Mathematical Programming,2011,127(1):3-30.
[24]Bouchaffra D,Vitae A,Cheriet M.Machine Learning and Pattern Recognition Models in Change Detection[J].Pattern Recognition,2015,48(3):613-615.
[25]Collins M,Schapire R E,Singer Y.Logistic Regression,Ada Boost and Bregman Distances[J].Machine Learning,2002,48(1):235-285.
[26]Canu S,Smola A.Kernel Methods and the Exponential Family[J].Neurocomputing,2006,69(7):714-720.
[27]Tsochantaridis I,Joachims T,Hofmann T,et al.Large Margin Methods for Structured and Interdependent Output Variables[J].Journal of Machine Learning Research,2005,1:1453-1484.
[28]Chang C C,Lin C J.LIBSVM:A Library for Support Vector Machines[J/OL].ACM Transactions on Intelligent Systems and Technology,2011,2(3).http://dx.doi.org/10.1145/1961189.1961199.