基于改进DBScan算法的雷暴挖掘与研究
|
摘要
为了从已发生的雷电现象中提取雷暴并研究雷暴活动规律。文中根据雷电定位系统采集的雷电数据,使用大数据、数据挖掘等计算机技术,开发基于Spark框架的改进DBScan算法,进行雷电聚类分析,提取雷暴,在ArcGIS软件中动态分析雷暴产生、爆发、移动、消亡全过程,最后通过计算机对雷暴进行分析统计,寻找雷暴活动规律。根据历史雷电数据逆向挖掘雷暴,掌握雷暴的活动规律,对今后雷暴预警奠定基础。
In order to extract thunderstorms from existing lightning phenomena and study thunderstorm activity law,based on the lightning data collected by the lightning location system, this paper develops an improved DBScan algorithm based on Spark framework, carries out lightning clustering analysis, extracts thunderstorms, dynamically analyzes the whole process of thunderstorms ' occurrence, outbreak, movement and extinction in ArcGIS software, and finally checks them by computer. Thunderstorms are analyzed statistically to find thunderstorm activity rules. According to the historical thunderstorm data, the thunderstorm is excavated in reverse, and the thunderstorm activity law is mastered, which lay a foundation for thunderstorm early warning in the future.
In order to extract thunderstorms from existing lightning phenomena and study thunderstorm activity law,based on the lightning data collected by the lightning location system, this paper develops an improved DBScan algorithm based on Spark framework, carries out lightning clustering analysis, extracts thunderstorms, dynamically analyzes the whole process of thunderstorms ' occurrence, outbreak, movement and extinction in ArcGIS software, and finally checks them by computer. Thunderstorms are analyzed statistically to find thunderstorm activity rules. According to the historical thunderstorm data, the thunderstorm is excavated in reverse, and the thunderstorm activity law is mastered, which lay a foundation for thunderstorm early warning in the future.
引文
[1]王蕾,钟思奕,曹华,等.雷电灾害监测与防雷堪察技术研究[J].气象水文海洋仪器,2018,25(1):112-117.WANG Lei, ZHONG Siyi, CAO Hua, el al. Resrarch on lightning disaster monitoring and lightning protection exploration techniques[J]. Meteorological, Hydrological and Marine Instruments, 2018,25(1):112-117.
[2]吴安坤.贵州省雷电灾害风险评价与区划研究[J].中国农业资源与区划,2018,39(2):88-93.WU Ankun. Research on risk assessment and zoning of lightning disasters in Guizhou province[J]. Chinese Journal of Agricultural Resources and Regional Planning, 2018,39(2):88-93.
[3]陈洪滨,朱彦良.雷暴探测研究的进展[J].大气科学,2012,36(2):411-422.CHEN Hongbin,ZHU Yanliang. Review on the observationinvestigati-on of thunderstorms[J].Chinese Journal of Atmospheric Sciences,2012,36(2):411-422.
[4]陈维江,陈家宏,谷山强,等.中国电网雷电监测与防护亟待研究的关键技术[J].高电压技术,2008, 34(10):2009-2015.CHEN Weijiang, CHEN Jiahong, GU Shanqiang, et al. Key Technologies of lightning detection and protection in china power grid[J]. High Voltage Engineering,2008,34(10):2009-2015.
[5]俞小鼎,周小刚,王秀明.雷暴与强对流临近天气预报技术进展[J].气象学报,2012, 70(3):311-337.YU Xiaoding, ZHOU Xiaogang, WANG Xiuming. The advances in the nowcasting techniques on thunderstorms and severs convection[J]. Acta Meteorologica Sinica,2012,70(3):311-337.
[6] SUN J G, LIU J,ZHAO L Y. Clustering algorithm research[J]. Journal of Software, 2018,19(1):48-61.
[7] ESTER M, KRIEGEL H P, SANDER J,et al. A densitybased algorithm for discovering clusters inlarge spatial databases with noise[C]//Kdd.[S.l.]:[s.n.],2017:226-231.
[8] JAIN A K,MURTY M N,FLYNN P J. Data clustering:A review[J]. ACM Computing Surveys(CSUR), 2017, 31(3):264-323.
[9] ZHANG T,RAMAKRISHNAN R,LIVNY M. BIRCH:An efficient data clustering method for very large databases[C]//ACM Sigmod Record.[S.1.]:ACM, 2017:103-114.
[10] WANG W, YANG J,MUNTZ R. STING:A statistical information-grid approach to spatialdata mining[C]//VLDB.[S.l.]:[s.n.],2016:186-195.
[11]仰孝富,齐建东,吉鹏飞,等.一种CF树结合KNN图划分的文本聚类算法[J].计算机工程与应用,2013,23(1):114-119.YANG Xiaofu, QI Jiangdong, JI Pengfei, et al.New text clustering algorithm based on CF tree and KNN graph partition[J]. Computer Engineering and Application, 2015,51(6):114-119.
[12]包晓安,鲍超,滕赛娜,等.基于DBSCAN算法的测试用例优化方法[J].计算机系统应用,2018,27(3):105-111.B AO Xiao'an, BAO Chao, TENG Saina, et al. Test case optimization method based on DBSCAN algorithm[J]. Computer Systems&Applications,2018,27(3):105-111.
[13]王佳玉,张振宇,褚佂,等.一种基于轨迹数据密度分区的分布式并行聚类方法[J].中国科学技术大学学报,2018, 48(1):47-56.WANG Jiayu, ZHANG Zhenyu, CHU Zheng, et al.A trajectory data density partition based distributed parallel clustering method[J]. Journal of University of Science and Technology of China,2018,48(1):47-56.
[14]王童童,荣垂田,卢卫,等.分布式图处理系统技术综述[J].软件学报,2018, 29(3):569-586.WANG Tongtong, RONG Chuitian, LU Wei, et al. Survey on technologies of distributed graph processing systems[J].Journal of Software,2018,29(3):569-586.
[15]周鹏飞,郭乔进,胡杰.基于Hadoop平台的视频处理系统设计[J].信息化研究,2016,42(6):64-67.ZHOU Pengfei, GUO Qiaojin, HU Jie. Video processing system based on hadoop platform[J]. Informatization Research,2016,42(6):64-67.
[16] ZAHARIA M,CHOWDHURY M,FRANKLIN MJ,etal.Spark:Cluster computing with working sets[C]//Hot Cloud, 2010.Boston,US:[s.n.],2017:10.
[17]冯万兴,朱晔,郭钧天,等.基于改进的DBSCAN方法和多项式拟合的雷电短时预测[J].计算机工程与科学,2014,36(10):2028-2033.FENG Wanxing, ZHU Ye, GUO Juntian, et al. Lightning forecast based on the improved DBSCAN and polynomial fitting[J]. Computer Engineering&Science, 2014, 36(10):2028-2033.
[18] HUFFINES G R, ORVILLE R E. Lightning ground flash density and thunderstorm duration in the continental United States:1989-1996[J]. Journal of Applied Meteorology, 1999,38(7):1013-1019.
[19]孙凌,周筠珺,郭在华.雷暴持续时间与地闪活动的预报方法研究[J].气象科学,2012,32(2):182-187.SUN Ling, ZHOU Yunjun, GUO Zaihua.Research on the forecasting method for thundrtstorm duration and lightning activities[J]. Journal of the Meteorological Sciences, 2012,32(2):182-187
[20]空军司令部气象研究室.国外关于积雨云特性的研究[J].气象科技,1975(10):4.Air Force Command Department of Meteorological Research. Foreign research on the characteristics of cumulonimbus clouds[J]. Meteorological Science and Technology,1975(10):4.
[21]刘希文,李得勤,韦惠红,等.2013年湖北地区两次雷暴大风过程环境及雷达回波特征[J].气象与环境学报,2018, 34(1):1-10.LIU Xiwen, LI Deqin, WEI Huihong, et al. Environmental and radar echoes characteristic analysis of two thunderstorm gale processes in Hubei province in 2013[J]. Journal of Meteorology and Environment,2018,34(1):1-10.
[22]冯玲,刘克剑,唐福喜,等.一种基于网格查询的改进DBSCAN算法[J].西华大学学报(自然科学版),2016,35(5):25-29.FENG Ling, LIU Kejian,TANG Fuxi,et al. Improvements of DBSCAN algorithm based on grid[J]. Journal of Xihua University(Natural Science),2016,35(5):25-29.
[23]张龙,汪新庆,刘岳,等.一种MapGIS向ArcGIS地图数据转换的方法[J].测绘科学,2018, 39(7):146-149.ZHANG Long, WANG Xinqing,LIU Yue,et al.A method of map data conversion from MapGIS into ArcGIS[J]. Science of Surveying and Mapping,2018,39(7):146-149.
[2]吴安坤.贵州省雷电灾害风险评价与区划研究[J].中国农业资源与区划,2018,39(2):88-93.WU Ankun. Research on risk assessment and zoning of lightning disasters in Guizhou province[J]. Chinese Journal of Agricultural Resources and Regional Planning, 2018,39(2):88-93.
[3]陈洪滨,朱彦良.雷暴探测研究的进展[J].大气科学,2012,36(2):411-422.CHEN Hongbin,ZHU Yanliang. Review on the observationinvestigati-on of thunderstorms[J].Chinese Journal of Atmospheric Sciences,2012,36(2):411-422.
[4]陈维江,陈家宏,谷山强,等.中国电网雷电监测与防护亟待研究的关键技术[J].高电压技术,2008, 34(10):2009-2015.CHEN Weijiang, CHEN Jiahong, GU Shanqiang, et al. Key Technologies of lightning detection and protection in china power grid[J]. High Voltage Engineering,2008,34(10):2009-2015.
[5]俞小鼎,周小刚,王秀明.雷暴与强对流临近天气预报技术进展[J].气象学报,2012, 70(3):311-337.YU Xiaoding, ZHOU Xiaogang, WANG Xiuming. The advances in the nowcasting techniques on thunderstorms and severs convection[J]. Acta Meteorologica Sinica,2012,70(3):311-337.
[6] SUN J G, LIU J,ZHAO L Y. Clustering algorithm research[J]. Journal of Software, 2018,19(1):48-61.
[7] ESTER M, KRIEGEL H P, SANDER J,et al. A densitybased algorithm for discovering clusters inlarge spatial databases with noise[C]//Kdd.[S.l.]:[s.n.],2017:226-231.
[8] JAIN A K,MURTY M N,FLYNN P J. Data clustering:A review[J]. ACM Computing Surveys(CSUR), 2017, 31(3):264-323.
[9] ZHANG T,RAMAKRISHNAN R,LIVNY M. BIRCH:An efficient data clustering method for very large databases[C]//ACM Sigmod Record.[S.1.]:ACM, 2017:103-114.
[10] WANG W, YANG J,MUNTZ R. STING:A statistical information-grid approach to spatialdata mining[C]//VLDB.[S.l.]:[s.n.],2016:186-195.
[11]仰孝富,齐建东,吉鹏飞,等.一种CF树结合KNN图划分的文本聚类算法[J].计算机工程与应用,2013,23(1):114-119.YANG Xiaofu, QI Jiangdong, JI Pengfei, et al.New text clustering algorithm based on CF tree and KNN graph partition[J]. Computer Engineering and Application, 2015,51(6):114-119.
[12]包晓安,鲍超,滕赛娜,等.基于DBSCAN算法的测试用例优化方法[J].计算机系统应用,2018,27(3):105-111.B AO Xiao'an, BAO Chao, TENG Saina, et al. Test case optimization method based on DBSCAN algorithm[J]. Computer Systems&Applications,2018,27(3):105-111.
[13]王佳玉,张振宇,褚佂,等.一种基于轨迹数据密度分区的分布式并行聚类方法[J].中国科学技术大学学报,2018, 48(1):47-56.WANG Jiayu, ZHANG Zhenyu, CHU Zheng, et al.A trajectory data density partition based distributed parallel clustering method[J]. Journal of University of Science and Technology of China,2018,48(1):47-56.
[14]王童童,荣垂田,卢卫,等.分布式图处理系统技术综述[J].软件学报,2018, 29(3):569-586.WANG Tongtong, RONG Chuitian, LU Wei, et al. Survey on technologies of distributed graph processing systems[J].Journal of Software,2018,29(3):569-586.
[15]周鹏飞,郭乔进,胡杰.基于Hadoop平台的视频处理系统设计[J].信息化研究,2016,42(6):64-67.ZHOU Pengfei, GUO Qiaojin, HU Jie. Video processing system based on hadoop platform[J]. Informatization Research,2016,42(6):64-67.
[16] ZAHARIA M,CHOWDHURY M,FRANKLIN MJ,etal.Spark:Cluster computing with working sets[C]//Hot Cloud, 2010.Boston,US:[s.n.],2017:10.
[17]冯万兴,朱晔,郭钧天,等.基于改进的DBSCAN方法和多项式拟合的雷电短时预测[J].计算机工程与科学,2014,36(10):2028-2033.FENG Wanxing, ZHU Ye, GUO Juntian, et al. Lightning forecast based on the improved DBSCAN and polynomial fitting[J]. Computer Engineering&Science, 2014, 36(10):2028-2033.
[18] HUFFINES G R, ORVILLE R E. Lightning ground flash density and thunderstorm duration in the continental United States:1989-1996[J]. Journal of Applied Meteorology, 1999,38(7):1013-1019.
[19]孙凌,周筠珺,郭在华.雷暴持续时间与地闪活动的预报方法研究[J].气象科学,2012,32(2):182-187.SUN Ling, ZHOU Yunjun, GUO Zaihua.Research on the forecasting method for thundrtstorm duration and lightning activities[J]. Journal of the Meteorological Sciences, 2012,32(2):182-187
[20]空军司令部气象研究室.国外关于积雨云特性的研究[J].气象科技,1975(10):4.Air Force Command Department of Meteorological Research. Foreign research on the characteristics of cumulonimbus clouds[J]. Meteorological Science and Technology,1975(10):4.
[21]刘希文,李得勤,韦惠红,等.2013年湖北地区两次雷暴大风过程环境及雷达回波特征[J].气象与环境学报,2018, 34(1):1-10.LIU Xiwen, LI Deqin, WEI Huihong, et al. Environmental and radar echoes characteristic analysis of two thunderstorm gale processes in Hubei province in 2013[J]. Journal of Meteorology and Environment,2018,34(1):1-10.
[22]冯玲,刘克剑,唐福喜,等.一种基于网格查询的改进DBSCAN算法[J].西华大学学报(自然科学版),2016,35(5):25-29.FENG Ling, LIU Kejian,TANG Fuxi,et al. Improvements of DBSCAN algorithm based on grid[J]. Journal of Xihua University(Natural Science),2016,35(5):25-29.
[23]张龙,汪新庆,刘岳,等.一种MapGIS向ArcGIS地图数据转换的方法[J].测绘科学,2018, 39(7):146-149.ZHANG Long, WANG Xinqing,LIU Yue,et al.A method of map data conversion from MapGIS into ArcGIS[J]. Science of Surveying and Mapping,2018,39(7):146-149.