基于散列查找和多线程调度的快速提取GRIB数据方法
|
摘要
GRIB码(GRIdded Binary)是WMO建议使用的一种表格驱动码,是与计算机无关的压缩的二进制码,其具有较强的检索能力、可扩展性、压缩比率高等特点.其通常用于存储和传输大容量的格点数据,广泛应用于数值天气预报、海洋数据和历史气象数据等数值预报产品中.当需要使用数据时,会先对GRIB格式文件进行解码,而因其特点所致,解码往往需要耗费大量时间.又因其常常应用于应急响应分析评价系统的预处理模块中,尤其在核应急响应系统中,解码部分的耗时是非常大的,严重影响到整体系统的分析运算效率.针对这一问题,本文提出了一种新的基于散列查找和多线程调度的快速提取GRIB数据方法,可大幅提升解码的效率,命名为HP算法(Hash&Parallel).
GRIB code( GRIdded Binary) is a table-driven code recommended by WMO. It is a computerindependent compressed binary code with strong retrieval capability,scalability,and high compression ratio. It is commonly used to store and transmit large-capacity grid point data,and is widely used in numerical forecasting products such as numerical weather prediction,ocean data and historical meteorological data. When data is needed,the GRIB format file is decoded first,and because of its characteristics,decoding often takes a lot of time.Because it is often used in the pre-processing module of the emergency response analysis and evaluation system,especially in the nuclear emergency response system,the time consumption of the decoding part is very large,which seriously affects the analysis and calculation efficiency of the overall system. Aiming at this problem,this paper proposes a new method for fast extraction of GRIB data based on hash search and multi-thread scheduling,which can greatly improve the efficiency of decoding. It is named HP algorithm( Hash & Parallel).
GRIB code( GRIdded Binary) is a table-driven code recommended by WMO. It is a computerindependent compressed binary code with strong retrieval capability,scalability,and high compression ratio. It is commonly used to store and transmit large-capacity grid point data,and is widely used in numerical forecasting products such as numerical weather prediction,ocean data and historical meteorological data. When data is needed,the GRIB format file is decoded first,and because of its characteristics,decoding often takes a lot of time.Because it is often used in the pre-processing module of the emergency response analysis and evaluation system,especially in the nuclear emergency response system,the time consumption of the decoding part is very large,which seriously affects the analysis and calculation efficiency of the overall system. Aiming at this problem,this paper proposes a new method for fast extraction of GRIB data based on hash search and multi-thread scheduling,which can greatly improve the efficiency of decoding. It is named HP algorithm( Hash & Parallel).
引文
[1]但玻,冯汉中,罗可生. ECMWF 0. 25*0. 25经纬网格模式资料处理及软件实现[J].高原山地气象研究,2013,33(3):92~96.
[2]姜亚飞.民航气象资料处理系统的并行设计与实现[D].河南大学,2015.
[3]张苈. ECMWF GRIB API及其应用[C].中国气象学会气象通信与信息技术委员会、国家气象信息中心. 2011年中国气象学会气象通信与信息技术委员会暨国家气象信息中心科技年会论文摘要[C].中国气象学会气象通信与信息技术委员会、国家气象信息中心,2011. 8.
[4]王建荣,华连生,唐怀瓯,等.数值预报产品分布式处理与存储系统设计[J].计算机技术与发展,2018,28(2):167~172.
[5]严蔚敏,吴伟民.数据结构(C语言版)[M].北京:清华大学出版社,2009.
[6] William Stallings.操作系统:精髓与设计原理[M].北京:机械工业出版社,2010.
[7]李冰,陈晓秋,乔清党,等.境外核事故放射性后果评价软件(RADCON)的开发及应用[J].辐射防护,2014,34(2):109~117.
[8]刘媛媛,应显勋,赵芳. GRIB2介绍及解码初探[J].气象科技,2006,(S1):61~64.
[9]党兰学,侯彦娥. GRIB数据中气象要素等值线的绘制[J].河南大学学报(自然科学版),2010,40(6):603~606.
[10]李建宏,何玉珠.多线程技术在复杂数据采集系统中的应用[J].电子测量技术,2008,(5):102~104.
[11]陈晓秋,李冰,余少青.日本福岛核事故对应急准备与响应工作的启示[J].辐射防护,2012,32(6):348~361,372.
[2]姜亚飞.民航气象资料处理系统的并行设计与实现[D].河南大学,2015.
[3]张苈. ECMWF GRIB API及其应用[C].中国气象学会气象通信与信息技术委员会、国家气象信息中心. 2011年中国气象学会气象通信与信息技术委员会暨国家气象信息中心科技年会论文摘要[C].中国气象学会气象通信与信息技术委员会、国家气象信息中心,2011. 8.
[4]王建荣,华连生,唐怀瓯,等.数值预报产品分布式处理与存储系统设计[J].计算机技术与发展,2018,28(2):167~172.
[5]严蔚敏,吴伟民.数据结构(C语言版)[M].北京:清华大学出版社,2009.
[6] William Stallings.操作系统:精髓与设计原理[M].北京:机械工业出版社,2010.
[7]李冰,陈晓秋,乔清党,等.境外核事故放射性后果评价软件(RADCON)的开发及应用[J].辐射防护,2014,34(2):109~117.
[8]刘媛媛,应显勋,赵芳. GRIB2介绍及解码初探[J].气象科技,2006,(S1):61~64.
[9]党兰学,侯彦娥. GRIB数据中气象要素等值线的绘制[J].河南大学学报(自然科学版),2010,40(6):603~606.
[10]李建宏,何玉珠.多线程技术在复杂数据采集系统中的应用[J].电子测量技术,2008,(5):102~104.
[11]陈晓秋,李冰,余少青.日本福岛核事故对应急准备与响应工作的启示[J].辐射防护,2012,32(6):348~361,372.