气象雷达原始回波数据无损压缩算法研究
|
摘要
气象雷达原始回波数据是雷达获取的原始资料,不仅可以用来检验雷达产品算法的准确性,而且反映了大气层动态变化的信息,是科学研究的宝贵资料。但随着气象雷达使用频率不断提高,使用范围不断扩大,雷达原始回波数据量巨增,因此对雷达原始回波进行无损数据压缩是非常有意义的。
本文主要针对气象雷达原始回波数据的特点,设计相应的无损数据压缩算法。首先,对气象雷达原始回波数据进行无损压缩研究的意义进行了综述,并对其研究概况进行了总结。然后,重点介绍了三类无损数据压缩算法的原理,包括:基于统计模型的压缩算法、基于字典模型的压缩算法以及文本数据压缩算法。其次,从统计分布和相关特性两个方面对实测雷达原始回波数据的特点进行了分析。接着,根据雷达原始回波数据的统计特性,并针对其存储类型的特点,设计了三类无损数据压缩算法。
第一类,文本类型数据的压缩算法。包括:位图压缩+Huffman压缩算法、半字节压缩+Huffman压缩算法和位图压缩+半字节压缩+Huffman压缩算法。第二类,二进制类型数据的压缩算法。包括:双Huffman压缩算法、位图压缩+Huffman压缩算法以及0阈值的双位图压缩+双Huffman压缩算法。其中位图压缩+Huffman压缩算法又派生出四种混合压缩算法,即:位图压缩+Huffman压缩算法、位图压缩+双Huffman压缩算法、双位图压缩+Huffman压缩算法以及双位图压缩+双Huffman压缩算法。
第三类,转换处理后数据的压缩算法。包括:阈值为28-1的混合压缩算法和阈值为216-1的混合压缩算法。
最后,采用设计的压缩算法对实测的雷达原始回波数据在C环境下进行压缩和解压实验,并从压缩因子、压缩时间及解压时间三个方面,与几种通用压缩算法及压缩软件行了比较。实验结果表明,设计的压缩算法中有7种算法的压缩性能都优于通用压缩算法和软件。从而说明,采用本文设计的混合压缩算法对气象雷达原始回波数据进行无损压缩是非常有效的。
Weather radar raw data is the primitive data of weather radar. It can be used to validate the veracity of the algorithm for radar product and reflect the information of aerosphere dynamic change, and it is also valuable for scientific research. But the amount of raw data of radar is so large with the use of weather radar frequently, that it is necessary to compress the raw data losslessly.
Lossless compression methods are designed by the characteristic of weather radar raw data in the dissertation. Firstly, the overview and the signification of weather radar raw data compression without loss are summarized. Secondly, three principles of lossless compression methods are emphatically presented, including statistical modeling compression, dictionary-based compression and text-based compression. Thirdly, the property of the statistic and the correlation of real radar data are analyzed. Then, three kinds of lossless compression methods are designed according to the property of statistic and storage.
The first kind of method is text-based compression algorithm, including bit mapping compression & Huffman, half-byte packing & Huffman, bit mapping compression & half-byte packing & Huffman.
The second kind of method is binary system type data compression algorithm, including double Huffman, bit mapping compression & Huffman, double bit mapping compression & double Huffman based on zero threshold. Four kinds of integrated compression methods are derived from bit mapping & Huffman. They are bit mapping compression & Huffman, bit mapping compression & double Huffman, double bit mapping & Huffman, double bit mapping compression & double Huffman.
The third kind of method is compression algorithm for switch data, including integrated compression algorithm based on the threshold of 28-1 and 216-1.
Finally, these integrated compression methods are used to compress and decompress real raw data in C environment, and compared to traditional compression methods and popular compression software by compression factor, compression time and decompression time, the experimental result shows that the seven proposed methods are better than the traditional methods and the popular compression software, so the integrated compression methods are suitable to compress weather radar raw data losslessly.
本文主要针对气象雷达原始回波数据的特点,设计相应的无损数据压缩算法。首先,对气象雷达原始回波数据进行无损压缩研究的意义进行了综述,并对其研究概况进行了总结。然后,重点介绍了三类无损数据压缩算法的原理,包括:基于统计模型的压缩算法、基于字典模型的压缩算法以及文本数据压缩算法。其次,从统计分布和相关特性两个方面对实测雷达原始回波数据的特点进行了分析。接着,根据雷达原始回波数据的统计特性,并针对其存储类型的特点,设计了三类无损数据压缩算法。
第一类,文本类型数据的压缩算法。包括:位图压缩+Huffman压缩算法、半字节压缩+Huffman压缩算法和位图压缩+半字节压缩+Huffman压缩算法。第二类,二进制类型数据的压缩算法。包括:双Huffman压缩算法、位图压缩+Huffman压缩算法以及0阈值的双位图压缩+双Huffman压缩算法。其中位图压缩+Huffman压缩算法又派生出四种混合压缩算法,即:位图压缩+Huffman压缩算法、位图压缩+双Huffman压缩算法、双位图压缩+Huffman压缩算法以及双位图压缩+双Huffman压缩算法。
第三类,转换处理后数据的压缩算法。包括:阈值为28-1的混合压缩算法和阈值为216-1的混合压缩算法。
最后,采用设计的压缩算法对实测的雷达原始回波数据在C环境下进行压缩和解压实验,并从压缩因子、压缩时间及解压时间三个方面,与几种通用压缩算法及压缩软件行了比较。实验结果表明,设计的压缩算法中有7种算法的压缩性能都优于通用压缩算法和软件。从而说明,采用本文设计的混合压缩算法对气象雷达原始回波数据进行无损压缩是非常有效的。
Weather radar raw data is the primitive data of weather radar. It can be used to validate the veracity of the algorithm for radar product and reflect the information of aerosphere dynamic change, and it is also valuable for scientific research. But the amount of raw data of radar is so large with the use of weather radar frequently, that it is necessary to compress the raw data losslessly.
Lossless compression methods are designed by the characteristic of weather radar raw data in the dissertation. Firstly, the overview and the signification of weather radar raw data compression without loss are summarized. Secondly, three principles of lossless compression methods are emphatically presented, including statistical modeling compression, dictionary-based compression and text-based compression. Thirdly, the property of the statistic and the correlation of real radar data are analyzed. Then, three kinds of lossless compression methods are designed according to the property of statistic and storage.
The first kind of method is text-based compression algorithm, including bit mapping compression & Huffman, half-byte packing & Huffman, bit mapping compression & half-byte packing & Huffman.
The second kind of method is binary system type data compression algorithm, including double Huffman, bit mapping compression & Huffman, double bit mapping compression & double Huffman based on zero threshold. Four kinds of integrated compression methods are derived from bit mapping & Huffman. They are bit mapping compression & Huffman, bit mapping compression & double Huffman, double bit mapping & Huffman, double bit mapping compression & double Huffman.
The third kind of method is compression algorithm for switch data, including integrated compression algorithm based on the threshold of 28-1 and 216-1.
Finally, these integrated compression methods are used to compress and decompress real raw data in C environment, and compared to traditional compression methods and popular compression software by compression factor, compression time and decompression time, the experimental result shows that the seven proposed methods are better than the traditional methods and the popular compression software, so the integrated compression methods are suitable to compress weather radar raw data losslessly.
引文
[1] 王立华,蓝天飞.使用压缩算法实现雷达原始资料共享的技术方法.湖北气象,2003,32-34
[2] Cirillo, Francis R.Adaptive Compression Algorithm Results for Complex Synthetic Aperture Radar Data.Proceedings of SPIE-The International Society for Optical Engineering,v5095,2003, p26-32
[3] Magli E,Olmo G,Penna B.Wavelet-based compression of SAR raw data.IGARSS,Toronto, Canada,2002,No.2:24-28
[4] 秦蕾.高倍数 SAR 原始数据压缩算法研究:[硕士论文].北京:中国科学院电子研究所,2004
[5] 朱永明.合成孔径雷达原始数据压缩算法研究:[硕士论文].南京:南京航空航天大学,2004
[6] 关振红,朱兆达,朱岱寅.SAR 原始数据压缩算法比较.军事通信技术,2005.2,Vol.26, Supp:58-62
[7] Paul Eichel,Robert W Ives.Compression of complex-valued SAR images. IEEE Transactions on Image Processing,1999,8(10):1483-1487
[8] 张乐平.基于自适应编码的SAR浮点模图像数据压缩.计算机与现代化,2005,No.11:18 –20
[9] 胡晓新,王岩飞.基于小波包变换的 SAR 原始数据压缩.电子与信息学报,2006.8,Vol. 28,No.8:1476-1479
[10] 潘志刚,王岩飞.一种基于小波变换的高倍数 SAR 原始数据压缩算法.电子与信息学报,2006,Vol.28,No.10:1798-1882
[11] Pascazio V, Schitinzi G.Wavelet transform coding for SAR raw data compression. IGARSS, Hamburg, Germany, 1999, (4):2251-2253
[12] 关振红,朱兆达,朱岱寅.SAR 原始数据压缩技术研究.南京航空航天大学学报,2005.6,Vol.37,No.3:325-329
[13] 夏卫平,王贞松,韩冀中.基于块自适应量化的SAR复数图像实时压缩算法.中国图像图形学报,2006,Vol.11,No.8:1097-1112
[14] 艾未华,黄云仙,白雪.嵌入式零树编码在天气雷达回波数据压缩中的应用.计算机应用与软件.2004.Vol.21,No.11:56-57
[15] Yan Wei,Zhu Zhaoda,Wang Qingdong,Ma Ning,Some new techniques used in digital weather radar networking and composition , CIE International Conference of RadarProceedings,2001,p874-877
[16] 陈威,李凤昌,章建军.基于小波零树的雷达回波数据的压缩.计算机工程与应用,2004,217-219
[17] Mahapatra,Pravas R. Makkapati,Vishnu V.Studies on a high-compression technique for weather radar reflectivity data.2005 Fifth International Conference on Information Communications and Signal Processing,v 2005,p895-899
[18] Mahapatra,Pravas R.Makkapati,Vishnu V.Performance enhancement of a spline-based method for extreme compression of weather radar reflectivity data.Proceedings of the Fifth IEEE International Symposium on Signal Processing and Information Technology,2005,p 147-152
[19] 叶红,安东升.基于小波变换的卫星云图压缩方法.计算机工程与科学,2004,vol.26,No.10:60-65
[20] 马宁,朱福萌,尹志军,蒋林辉.改进游程编码在天气雷达数据压缩中的应用.解放军理工大学学报(自然科学版),2004,Vol.5,No.6:88-90
[21] MA Ning,YAN Wei,WANG Qingdong.Integer Wavelet Transform for Weather Radar Data Compression.Institute of Meteorology, Proceedings of SPIE ,2001,Vol.4551:225-228
[22] 郑朝晖.基于算术编码的雷达波形数据压缩.火控雷达技术,2002,Vol.31,No.2:24-29
[23] 袁玫等.数据压缩技术及应用.电子工业出版社:1995.94-114
[24] Tom Lookabaugh. (李熠辉,译者).多媒体数据压缩原理与标准.电子工业出版社,2000
[25] 吴乐南. 数据压缩.电子工业出版社:2000.20-54
[26] 黄妙珍,吴轶,居悌,用自适应 HUFFMAN 编码实现数据的压缩与解压 微机发展,1999. No.6-10
[27] David Salomon 著,(吴乐南等,译者).数据压缩原理与应用(第二版),北京:电子工业出版社,2003.1-8
[28] 何建新.现代天气雷达.电子科技大学出版社:2004.148-149
[29] 焦中生,沈超玲,张云.气象雷达原理 . 气象出版社: 2005.302-306
[30] 胡广军.数字信号处理-理论算法与实现.清华大学出版社:1997.22-25
[2] Cirillo, Francis R.Adaptive Compression Algorithm Results for Complex Synthetic Aperture Radar Data.Proceedings of SPIE-The International Society for Optical Engineering,v5095,2003, p26-32
[3] Magli E,Olmo G,Penna B.Wavelet-based compression of SAR raw data.IGARSS,Toronto, Canada,2002,No.2:24-28
[4] 秦蕾.高倍数 SAR 原始数据压缩算法研究:[硕士论文].北京:中国科学院电子研究所,2004
[5] 朱永明.合成孔径雷达原始数据压缩算法研究:[硕士论文].南京:南京航空航天大学,2004
[6] 关振红,朱兆达,朱岱寅.SAR 原始数据压缩算法比较.军事通信技术,2005.2,Vol.26, Supp:58-62
[7] Paul Eichel,Robert W Ives.Compression of complex-valued SAR images. IEEE Transactions on Image Processing,1999,8(10):1483-1487
[8] 张乐平.基于自适应编码的SAR浮点模图像数据压缩.计算机与现代化,2005,No.11:18 –20
[9] 胡晓新,王岩飞.基于小波包变换的 SAR 原始数据压缩.电子与信息学报,2006.8,Vol. 28,No.8:1476-1479
[10] 潘志刚,王岩飞.一种基于小波变换的高倍数 SAR 原始数据压缩算法.电子与信息学报,2006,Vol.28,No.10:1798-1882
[11] Pascazio V, Schitinzi G.Wavelet transform coding for SAR raw data compression. IGARSS, Hamburg, Germany, 1999, (4):2251-2253
[12] 关振红,朱兆达,朱岱寅.SAR 原始数据压缩技术研究.南京航空航天大学学报,2005.6,Vol.37,No.3:325-329
[13] 夏卫平,王贞松,韩冀中.基于块自适应量化的SAR复数图像实时压缩算法.中国图像图形学报,2006,Vol.11,No.8:1097-1112
[14] 艾未华,黄云仙,白雪.嵌入式零树编码在天气雷达回波数据压缩中的应用.计算机应用与软件.2004.Vol.21,No.11:56-57
[15] Yan Wei,Zhu Zhaoda,Wang Qingdong,Ma Ning,Some new techniques used in digital weather radar networking and composition , CIE International Conference of RadarProceedings,2001,p874-877
[16] 陈威,李凤昌,章建军.基于小波零树的雷达回波数据的压缩.计算机工程与应用,2004,217-219
[17] Mahapatra,Pravas R. Makkapati,Vishnu V.Studies on a high-compression technique for weather radar reflectivity data.2005 Fifth International Conference on Information Communications and Signal Processing,v 2005,p895-899
[18] Mahapatra,Pravas R.Makkapati,Vishnu V.Performance enhancement of a spline-based method for extreme compression of weather radar reflectivity data.Proceedings of the Fifth IEEE International Symposium on Signal Processing and Information Technology,2005,p 147-152
[19] 叶红,安东升.基于小波变换的卫星云图压缩方法.计算机工程与科学,2004,vol.26,No.10:60-65
[20] 马宁,朱福萌,尹志军,蒋林辉.改进游程编码在天气雷达数据压缩中的应用.解放军理工大学学报(自然科学版),2004,Vol.5,No.6:88-90
[21] MA Ning,YAN Wei,WANG Qingdong.Integer Wavelet Transform for Weather Radar Data Compression.Institute of Meteorology, Proceedings of SPIE ,2001,Vol.4551:225-228
[22] 郑朝晖.基于算术编码的雷达波形数据压缩.火控雷达技术,2002,Vol.31,No.2:24-29
[23] 袁玫等.数据压缩技术及应用.电子工业出版社:1995.94-114
[24] Tom Lookabaugh. (李熠辉,译者).多媒体数据压缩原理与标准.电子工业出版社,2000
[25] 吴乐南. 数据压缩.电子工业出版社:2000.20-54
[26] 黄妙珍,吴轶,居悌,用自适应 HUFFMAN 编码实现数据的压缩与解压 微机发展,1999. No.6-10
[27] David Salomon 著,(吴乐南等,译者).数据压缩原理与应用(第二版),北京:电子工业出版社,2003.1-8
[28] 何建新.现代天气雷达.电子科技大学出版社:2004.148-149
[29] 焦中生,沈超玲,张云.气象雷达原理 . 气象出版社: 2005.302-306
[30] 胡广军.数字信号处理-理论算法与实现.清华大学出版社:1997.22-25