海量监测数据下分布式BP神经网络区域滑坡空间预测方法
|
摘要
提出BP神经网络的分布式区域滑坡预测方法,算法设计在大数据分布式处理平台Spark下实现,通过构造包含均方误差和L2正则化的代价函数,提高运算实时性和算法泛化能力。统计影响滑坡评价因子的量化指标和定义监测剖面危险级别评价值,并进行评价因子特征选取,用于三峡库区忠县区域9个滑坡11年月监测海量数据挖掘,对研究区所有滑坡监测剖面每月进行危险级别评价,实现以月为周期的区域滑坡危险程度空间预测。试验表明,采用所述方法得到的拟合精度、准确度、效率均比梯度提升决策树、随机森林算法好,预测的滑坡危险级别准确,该方法可作为区域滑坡空间预测的一种新思路。
Landslides have characteristics such as regionality, multipleness, and seriousness. The traditional area landslide spatial prediction method, under massive data condition, has poor real-time performance and strong subjectivity, and the evaluation performance is poor under multiple factors. A distributed regional landslide prediction method based on BP neural network is proposed in this paper. The algorithm is designed as a parallel computing environment of big data processing platform Spark, and the cost function of BP network is designed as two items of mean square error and L2 regularization, which improves generalization ability. Through statistics of the quantitative indicators of landslide factors and the definition of hazard index of monitoring profile, the influencing factors are selected. This approach is applied to massive data mining of 9 landslides in 11 years in Zhongxian area of Three Gorges Reservoir area, which achieves the combination of qualitative analysis and quantitative analysis. All the landslide monitoring sections in the study area were monthly evaluated to determine the risk level, and the spatial prediction of the monthly landslide risk degree was achieved. Experiments show that the fitting accuracy and efficiency obtained by the method are better than gradient-based decision trees and random forest algorithms, and the prediction area landslide risk assessment accuracy is good. This method can be used as a new approach for regional landslide spatial prediction.
Landslides have characteristics such as regionality, multipleness, and seriousness. The traditional area landslide spatial prediction method, under massive data condition, has poor real-time performance and strong subjectivity, and the evaluation performance is poor under multiple factors. A distributed regional landslide prediction method based on BP neural network is proposed in this paper. The algorithm is designed as a parallel computing environment of big data processing platform Spark, and the cost function of BP network is designed as two items of mean square error and L2 regularization, which improves generalization ability. Through statistics of the quantitative indicators of landslide factors and the definition of hazard index of monitoring profile, the influencing factors are selected. This approach is applied to massive data mining of 9 landslides in 11 years in Zhongxian area of Three Gorges Reservoir area, which achieves the combination of qualitative analysis and quantitative analysis. All the landslide monitoring sections in the study area were monthly evaluated to determine the risk level, and the spatial prediction of the monthly landslide risk degree was achieved. Experiments show that the fitting accuracy and efficiency obtained by the method are better than gradient-based decision trees and random forest algorithms, and the prediction area landslide risk assessment accuracy is good. This method can be used as a new approach for regional landslide spatial prediction.
引文
[1]田维刚,邓红卫,雷涛,等.区域滑坡稳定性评价系统的设计与实现[J].防灾减灾工程学报,2013,33(2):155-161.TIAN Wei-gang,DENG Hong-wei,LEI Tao,et al.Design and implementation of regional landslide stability evaluation system[J].Journal of Disaster Prevention and Mitigation Engineering,2013,33(2):155-161.
[2]BOUALLA O,MEHDI K,FADILI A,et al.GIS-based landslide susceptibility mapping in the Safi region,West Morocco[J].Bulletin of Engineering Geology&the Environment,2017,76(4):1-18.
[3]桂蕾,殷坤龙,王佳佳.基于聚类分析的滑坡灾害危险性区划研究[J].水文地质工程地质,2013,40(1):100-105.GUI Lei,YIN Kun-long,WANG Jia-jia.Landslide hazard zonation based on cluster analysis[J].Hydrogeology&Engineering Geology,2013,40(1):100-105.
[4]王佳佳,殷坤龙,肖莉丽.基于GIS和信息量的滑坡灾害易发性评价-以三峡库区万州区为例[J].岩石力学与工程学报,2014,33(4):797-808.WANG Jia-jia,YIN Kun-long,XIAO Li-li.Landslide susceptibility assessment based on GIS and weighted information value:a case study of Wanzhou district,Three Gorges Reservoir[J].Chinese Journal of Rock Mechanics and Engineering,2014,33(4):797-808.
[5]邓冬梅,梁烨,王亮清,等.基于集合经验模态分解与支持向量机回归的位移预测方法:以三峡库区滑坡为例[J].岩土力学,2017,38(12):3660-3669.DENG Dong-mei,LIANG Ye,WANG Liang-qing,et al.Displacement prediction method based on ensemble empirical mode decomposition and support vector machine regression-a case of landslides in Three Gorges reservoir area[J].Rock and Soil Mechanics,2017,38(12):3660-3669.
[6]CAN A,DAGDELENLER G,ERCANOGLU M,et al.Landslide susceptibility mapping at Ovac?k-Karabük(Turkey)using different artificial neural network models:comparison of training algorithms[J].Bulletin of Engineering Geology&the Environment,2017,78(1):1-14.
[7]王志旺,李端有,王湘桂.区域滑坡空间预测方法研究综述[J].长江科学院院报,2012,29(5):78-85.WANG Zhi-wang,LI Duan-you,WANG Xiang-gui.Areview of spatial prediction methods for regional landslides[J].Journal of Yangtze River Scientific Research Institute,2012,29(5):78-85.
[8]DEAN J,GHEMAWAT S.Map Reduce:simplified data processing on large clusters[M].[S.l.]:ACM,2008.
[9]SETHI K K,RAMESH D.HFIM:a spark-based hybrid frequent itemset mining algorithm for big data processing[J].The Journal of Supercomputing,2017,73(8):3652-3668.
[10]刘磊,殷坤龙,徐勇,等.考虑降雨及库水位变动的区域滑坡灾害稳定性评价研究[J].岩石力学与工程学报,2018,37(2):403-414.LIU Lei,YIN Kun-long,XU Yong.Evaluation of regional landslide stability considering rainfall and variation of water level of reservoir[J].Chinese Journal of Rock Mechanics and Engineering,2018,37(2):403-414.
[11]HUANG S,LI C,LUO L.Landslide monitoring network establishment within unified datum and stability analysis in the Three Gorges reservoir area[J].Journal of Sensors,2017(9):1-13.
[12]PAL A,AGRAWAL S.An experimental approach towards big data for analyzing memory utilization on a hadoop cluster using HDFS and Map Reduce[C]//First International Conference on Networks&Soft Computing.[S.l.]:IEEE Press,2014.
[13]WU X,ZHAN F B,ZHANG K,et al.Application of a two-step cluster analysis and the Apriori algorithm to classify the deformation states of two typical colluvial landslides in the Three Gorges,China[J].Environmental Earth Sciences,2016,75(2):146.
[14]张艳博,杨震,姚旭龙,等.基于声发射信号聚类分析和神经网络识别的岩爆预警方法实验研究[J].岩土力学,2017,38(增刊2):89-98.ZHANG Yan-bo,YANG Zhen,YAO Xu-long,et al.Experimental study of rockburst early warning method based on acoustic emission cluster analysis and neural network identification[J].Rock and Soil Mechanics,2017,38(Suppl.2):89-98.
[15]HECHT-NIELSEN R.Kolmogorov’s mapping neural network existence theorem[C]//IEEE International Conference on Neural Networks.[S.l.]:IEEE.Press,1987.
[16]武妍,张立明.神经网络的泛化能力与结构优化算法研究[J].计算机应用研究,2002,19(6):21-25.WU Yan,ZHANG Li-ming.A survey of research work on neural network generalization and structure optimization algorithms[J].Application Research of Computers,2002,19(6):21-25.
[17]叶倩怡.基于Xgboost方法的实体零售业销售额预测研究[D].南昌:南昌大学,2016.YE Qian-yi.Study on sales forecasts of stores based on Xgboost method[D].Nanchang:Nanchang University,2016.
[18]李朋丽,田伟平,李家春.基于BP神经网络的滑坡稳定性分析[J].广西大学学报(自然科学版),2013,38(4):905-911.LI Peng-li,TIAN Wei-ping,LI Jia-chun.Analysis of landslide stability based on BP neural network[J].Journal of Guangxi University(Natural Science Edition),2013,38(4):905-911.
[19]柯国林.梯度提升决策树(GBTS)并行学习算法研究[D].厦门:厦门大学,2016.KE Guo-lin.Research on gradient boost decision tree(GBDT)parallel learning algorithm[D].Xiamen:Xiamen University,2016.
[20]PAL M.Random forest classifier for remote sensing classification[J].International Journal of Remote Sensing,2005,26(1):217-222.
[2]BOUALLA O,MEHDI K,FADILI A,et al.GIS-based landslide susceptibility mapping in the Safi region,West Morocco[J].Bulletin of Engineering Geology&the Environment,2017,76(4):1-18.
[3]桂蕾,殷坤龙,王佳佳.基于聚类分析的滑坡灾害危险性区划研究[J].水文地质工程地质,2013,40(1):100-105.GUI Lei,YIN Kun-long,WANG Jia-jia.Landslide hazard zonation based on cluster analysis[J].Hydrogeology&Engineering Geology,2013,40(1):100-105.
[4]王佳佳,殷坤龙,肖莉丽.基于GIS和信息量的滑坡灾害易发性评价-以三峡库区万州区为例[J].岩石力学与工程学报,2014,33(4):797-808.WANG Jia-jia,YIN Kun-long,XIAO Li-li.Landslide susceptibility assessment based on GIS and weighted information value:a case study of Wanzhou district,Three Gorges Reservoir[J].Chinese Journal of Rock Mechanics and Engineering,2014,33(4):797-808.
[5]邓冬梅,梁烨,王亮清,等.基于集合经验模态分解与支持向量机回归的位移预测方法:以三峡库区滑坡为例[J].岩土力学,2017,38(12):3660-3669.DENG Dong-mei,LIANG Ye,WANG Liang-qing,et al.Displacement prediction method based on ensemble empirical mode decomposition and support vector machine regression-a case of landslides in Three Gorges reservoir area[J].Rock and Soil Mechanics,2017,38(12):3660-3669.
[6]CAN A,DAGDELENLER G,ERCANOGLU M,et al.Landslide susceptibility mapping at Ovac?k-Karabük(Turkey)using different artificial neural network models:comparison of training algorithms[J].Bulletin of Engineering Geology&the Environment,2017,78(1):1-14.
[7]王志旺,李端有,王湘桂.区域滑坡空间预测方法研究综述[J].长江科学院院报,2012,29(5):78-85.WANG Zhi-wang,LI Duan-you,WANG Xiang-gui.Areview of spatial prediction methods for regional landslides[J].Journal of Yangtze River Scientific Research Institute,2012,29(5):78-85.
[8]DEAN J,GHEMAWAT S.Map Reduce:simplified data processing on large clusters[M].[S.l.]:ACM,2008.
[9]SETHI K K,RAMESH D.HFIM:a spark-based hybrid frequent itemset mining algorithm for big data processing[J].The Journal of Supercomputing,2017,73(8):3652-3668.
[10]刘磊,殷坤龙,徐勇,等.考虑降雨及库水位变动的区域滑坡灾害稳定性评价研究[J].岩石力学与工程学报,2018,37(2):403-414.LIU Lei,YIN Kun-long,XU Yong.Evaluation of regional landslide stability considering rainfall and variation of water level of reservoir[J].Chinese Journal of Rock Mechanics and Engineering,2018,37(2):403-414.
[11]HUANG S,LI C,LUO L.Landslide monitoring network establishment within unified datum and stability analysis in the Three Gorges reservoir area[J].Journal of Sensors,2017(9):1-13.
[12]PAL A,AGRAWAL S.An experimental approach towards big data for analyzing memory utilization on a hadoop cluster using HDFS and Map Reduce[C]//First International Conference on Networks&Soft Computing.[S.l.]:IEEE Press,2014.
[13]WU X,ZHAN F B,ZHANG K,et al.Application of a two-step cluster analysis and the Apriori algorithm to classify the deformation states of two typical colluvial landslides in the Three Gorges,China[J].Environmental Earth Sciences,2016,75(2):146.
[14]张艳博,杨震,姚旭龙,等.基于声发射信号聚类分析和神经网络识别的岩爆预警方法实验研究[J].岩土力学,2017,38(增刊2):89-98.ZHANG Yan-bo,YANG Zhen,YAO Xu-long,et al.Experimental study of rockburst early warning method based on acoustic emission cluster analysis and neural network identification[J].Rock and Soil Mechanics,2017,38(Suppl.2):89-98.
[15]HECHT-NIELSEN R.Kolmogorov’s mapping neural network existence theorem[C]//IEEE International Conference on Neural Networks.[S.l.]:IEEE.Press,1987.
[16]武妍,张立明.神经网络的泛化能力与结构优化算法研究[J].计算机应用研究,2002,19(6):21-25.WU Yan,ZHANG Li-ming.A survey of research work on neural network generalization and structure optimization algorithms[J].Application Research of Computers,2002,19(6):21-25.
[17]叶倩怡.基于Xgboost方法的实体零售业销售额预测研究[D].南昌:南昌大学,2016.YE Qian-yi.Study on sales forecasts of stores based on Xgboost method[D].Nanchang:Nanchang University,2016.
[18]李朋丽,田伟平,李家春.基于BP神经网络的滑坡稳定性分析[J].广西大学学报(自然科学版),2013,38(4):905-911.LI Peng-li,TIAN Wei-ping,LI Jia-chun.Analysis of landslide stability based on BP neural network[J].Journal of Guangxi University(Natural Science Edition),2013,38(4):905-911.
[19]柯国林.梯度提升决策树(GBTS)并行学习算法研究[D].厦门:厦门大学,2016.KE Guo-lin.Research on gradient boost decision tree(GBDT)parallel learning algorithm[D].Xiamen:Xiamen University,2016.
[20]PAL M.Random forest classifier for remote sensing classification[J].International Journal of Remote Sensing,2005,26(1):217-222.