大面积水深异常检测的条件变化自编码算法
|
摘要
针对大面积海底地形数据缺失或异常的复杂及多变性特点,结合条件变分自编码器(CVAE)与深度卷积生成对抗网络(DCGAN),构建了条件变分自编码生成对抗网络(CVAE-GAN)大面积海底伪地形的检测与剔除方法。本文方法利用条件变分自编码算法改变原有的样本分布,通过对训练样本的学习重新构建样本之间的分布规律,有效提高了高维到低维映射的稳定性;结合生成对抗网络,提高了整体算法的稳健性,最终得到较优的检测与剔除结果。采用水深格网数据进行试验,并与中值滤波法、趋势面滤波法进行比较。结果表明,本文方法在精度、稳定性及噪声稳健性方面有所提高,验证了本文方法在海底地形数据处理上具有可行性。
In view of the complexity and variability of bathymetric data missing and exception, an algorithm named CVAE-GAN to detect and eliminate the widespread bathymetric outliersis proposed. Firstly, the proposed model is an alternative to traditional generative adversarial network(GAN) training methods, combined with the advantages of conditional variational autoencoder(CVAE) and deep convolutional generative adversarial network(DCGAN).Secondly, the network structure is introduced in detail.The generalized CVAE algorithm is added to change and reshape the sample distribution, having a better ability of dimensionality reduction.The GAN method improves the robustness of the whole algorithm.Thirdly,using electronic chart data containing widespread outliers, long-time experiments were carried out to train the CVAE-GAN till optimality. Finally, compared with median filtering method and trend filtering algorithm(TFA), the results show that the proposed method has an improvement in accuracy, stability and robustness.It is also verified that the feasibility of the proposedmethod in bathymetric data processing.
In view of the complexity and variability of bathymetric data missing and exception, an algorithm named CVAE-GAN to detect and eliminate the widespread bathymetric outliersis proposed. Firstly, the proposed model is an alternative to traditional generative adversarial network(GAN) training methods, combined with the advantages of conditional variational autoencoder(CVAE) and deep convolutional generative adversarial network(DCGAN).Secondly, the network structure is introduced in detail.The generalized CVAE algorithm is added to change and reshape the sample distribution, having a better ability of dimensionality reduction.The GAN method improves the robustness of the whole algorithm.Thirdly,using electronic chart data containing widespread outliers, long-time experiments were carried out to train the CVAE-GAN till optimality. Finally, compared with median filtering method and trend filtering algorithm(TFA), the results show that the proposed method has an improvement in accuracy, stability and robustness.It is also verified that the feasibility of the proposedmethod in bathymetric data processing.
引文
[1] 何义斌,吴书帮,谢洪燕,等.多波束异常测深数据检测方法实践[J].测绘科学,2004,29(1):50-52.HE Yibin,WU Shubang,XIE Hongyan,et al.Study of detection and filter of outlier on the sounding data of MES[J].Science of Surveying and Mapping,2004,29(1):50-52.
[2] 翟国君,黄谟涛.海洋测量技术研究进展与展望[J].测绘学报,2017,46(10):1752-1759.DOI:10.11947/j.AGCS.2017.20170309.ZHAI Guojun,HUANG Motao.The review of development of marine surveying technology[J].Acta Geodaetica et Cartographica Sinica,2017,46(10):1752-1759.DOI:10.11947/j.AGCS.2017.20170309.
[3] 李家彪,王小波,华祖根,等.多波束勘测原理技术与方法[M].北京:海洋出版社,1999:197-207.LI Jiabiao,WANG Xiaobo,HUA Zugen,et al.Principles and methods of multibeam surveying[M].Beijing:China Ocean Press,1999:197-207.
[4] 赵建虎,刘经南.多波束测深及图像数据处理[M].武汉:武汉大学出版社,2008.ZHAO Jianhu,LIU Jingnan.Multibeam echosounding and image data processing[M].Wuhan:Wuhan University Press,2008.
[5] 刘雁春.海洋测深空间结构及其数据处理[M].北京:测绘出版社,2003.LIU Yanchun.Spatial structure of bathymetry and its data processing[M].Beijing:Surveying and Mapping Press,2003.
[6] HERLIHY D R,STEPKA T N,RULON T D.Filtering erroneous soundings from multibeam survey data[J].International Hydrographic Review,1992,69(2):67-76.
[7] WARE C.A system for cleaning high volume bathymetry[J].International Hydrographic Review,1992,69(2):77-94.
[8] SHAW S,ARNOLD J.Automated error detection in multibeam bathymetry data[C]//Proceedings of the Ocean'93.Victoria,BC,Canada:IEEE,1993(2):II/89-II/94.
[9] EEG J.On the identification of spikes in soundings[J].International Hydrographic Review,2015,LXXII(1):33-41.
[10] LIRAKIS C B,BONGIOVANNI K P.Automated multibeam data cleaning and target detection[C]//Proceedings of the Oceans 2000 MTS/IEEE Conference and Exhibition.Conference Proceedings.Providence,RI,USA:IEEE,2000:719-723.
[11] MANN M,AGATHOKLIS P,ANTONIOU A.Automatic outlier detection in multibeam data using median filtering[C]//Proceedings of 2001 IEEE Pacific Rim Conference on Communications,Computers and Signal Processing.Victoria,BC,Canada:IEEE,2001:690-693.
[12] CALDER B R,MAYER L A.Robust automatic multi-beam bathymetric processing[C]//Proceedings of the Center for Coastal and Ocean Mapping.Norfolk,VA,USA:Hydrographic Society of America,2001.
[13] JAKOBSSON M,CALDER B,MAYER L.On the effect of random errors in gridded bathymetric compilations[J].Journal of Geophysical Research:Solid Earth,2002,107(B12):ETG 14-1-ETG 14-11.
[14] CALDER B R,MAYER L A.Automatic processing of high-rate,high-density multibeam echosounder data[J].Geochemistry,Geophysics,Geosystems,2003,4(6):24-48.
[15] GOODFELLOW I J,POUGETABADIE J,MIRZA M,et al.Generative adversarial networks[J].Advances in Neural Information Processing Systems,2014(3):2672-2680.
[16] GOODFELLOW I,BENGIO Y,COURVILLE A.Deep learning[M].Cambridge:The MIT Press,2016.
[17] RADFORD A,METZ L,CHINTALA S.Unsupervised representation learning with deep convolutional generative adversarial networks[J].arXiv preprint arXiv:1511.06434,2015.
[18] DENTON E,CHINTALA S,SZLAM A,et al.Deep generative image models using a laplacian pyramid of adversarial networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems.Montreal,Canada:MIT Press,2015.
[19] MIRZA M,OSINDERO S.Conditional generative adversarial nets[J].arXiv preprint arXiv:1411.1784,2014.
[20] 李金洪,黄永昌.深度学习之TensorFlow入门、原理与进阶实战[M].北京:机械工业出版社,2018.LI Jinhong,HUANG Yongchang.Getting started and best practices with tensorFlow for deep learning[M].Beijing:China Machine Press,2018.
[21] WELLING M,ROSEN-ZVI M,HINTON G E.Exponential family harmoniums with an application to information retrieval[C]//Proceedings of the 17th International Conference on Neural Information Processing Systems.Vancouver,British Columbia,Canada:MIT Press,2004.
[22] VINCENT P.A connection between score matching and denoising autoencoders[J].Neural Computation,2011,23(7):1661-1674.
[23] DUCHI J,HAZAN E,SINGER Y.Adaptive subgradient methods for online learning and stochastic optimization[J].Journal of Machine Learning Research,2011(12):2121-2159.
[2] 翟国君,黄谟涛.海洋测量技术研究进展与展望[J].测绘学报,2017,46(10):1752-1759.DOI:10.11947/j.AGCS.2017.20170309.ZHAI Guojun,HUANG Motao.The review of development of marine surveying technology[J].Acta Geodaetica et Cartographica Sinica,2017,46(10):1752-1759.DOI:10.11947/j.AGCS.2017.20170309.
[3] 李家彪,王小波,华祖根,等.多波束勘测原理技术与方法[M].北京:海洋出版社,1999:197-207.LI Jiabiao,WANG Xiaobo,HUA Zugen,et al.Principles and methods of multibeam surveying[M].Beijing:China Ocean Press,1999:197-207.
[4] 赵建虎,刘经南.多波束测深及图像数据处理[M].武汉:武汉大学出版社,2008.ZHAO Jianhu,LIU Jingnan.Multibeam echosounding and image data processing[M].Wuhan:Wuhan University Press,2008.
[5] 刘雁春.海洋测深空间结构及其数据处理[M].北京:测绘出版社,2003.LIU Yanchun.Spatial structure of bathymetry and its data processing[M].Beijing:Surveying and Mapping Press,2003.
[6] HERLIHY D R,STEPKA T N,RULON T D.Filtering erroneous soundings from multibeam survey data[J].International Hydrographic Review,1992,69(2):67-76.
[7] WARE C.A system for cleaning high volume bathymetry[J].International Hydrographic Review,1992,69(2):77-94.
[8] SHAW S,ARNOLD J.Automated error detection in multibeam bathymetry data[C]//Proceedings of the Ocean'93.Victoria,BC,Canada:IEEE,1993(2):II/89-II/94.
[9] EEG J.On the identification of spikes in soundings[J].International Hydrographic Review,2015,LXXII(1):33-41.
[10] LIRAKIS C B,BONGIOVANNI K P.Automated multibeam data cleaning and target detection[C]//Proceedings of the Oceans 2000 MTS/IEEE Conference and Exhibition.Conference Proceedings.Providence,RI,USA:IEEE,2000:719-723.
[11] MANN M,AGATHOKLIS P,ANTONIOU A.Automatic outlier detection in multibeam data using median filtering[C]//Proceedings of 2001 IEEE Pacific Rim Conference on Communications,Computers and Signal Processing.Victoria,BC,Canada:IEEE,2001:690-693.
[12] CALDER B R,MAYER L A.Robust automatic multi-beam bathymetric processing[C]//Proceedings of the Center for Coastal and Ocean Mapping.Norfolk,VA,USA:Hydrographic Society of America,2001.
[13] JAKOBSSON M,CALDER B,MAYER L.On the effect of random errors in gridded bathymetric compilations[J].Journal of Geophysical Research:Solid Earth,2002,107(B12):ETG 14-1-ETG 14-11.
[14] CALDER B R,MAYER L A.Automatic processing of high-rate,high-density multibeam echosounder data[J].Geochemistry,Geophysics,Geosystems,2003,4(6):24-48.
[15] GOODFELLOW I J,POUGETABADIE J,MIRZA M,et al.Generative adversarial networks[J].Advances in Neural Information Processing Systems,2014(3):2672-2680.
[16] GOODFELLOW I,BENGIO Y,COURVILLE A.Deep learning[M].Cambridge:The MIT Press,2016.
[17] RADFORD A,METZ L,CHINTALA S.Unsupervised representation learning with deep convolutional generative adversarial networks[J].arXiv preprint arXiv:1511.06434,2015.
[18] DENTON E,CHINTALA S,SZLAM A,et al.Deep generative image models using a laplacian pyramid of adversarial networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems.Montreal,Canada:MIT Press,2015.
[19] MIRZA M,OSINDERO S.Conditional generative adversarial nets[J].arXiv preprint arXiv:1411.1784,2014.
[20] 李金洪,黄永昌.深度学习之TensorFlow入门、原理与进阶实战[M].北京:机械工业出版社,2018.LI Jinhong,HUANG Yongchang.Getting started and best practices with tensorFlow for deep learning[M].Beijing:China Machine Press,2018.
[21] WELLING M,ROSEN-ZVI M,HINTON G E.Exponential family harmoniums with an application to information retrieval[C]//Proceedings of the 17th International Conference on Neural Information Processing Systems.Vancouver,British Columbia,Canada:MIT Press,2004.
[22] VINCENT P.A connection between score matching and denoising autoencoders[J].Neural Computation,2011,23(7):1661-1674.
[23] DUCHI J,HAZAN E,SINGER Y.Adaptive subgradient methods for online learning and stochastic optimization[J].Journal of Machine Learning Research,2011(12):2121-2159.