基于生成对抗网络的单帧遥感图像超分辨率
|
摘要
受成像设备、传输条件等因素限制,遥感图像的清晰度难以保证。图像超分辨率技术旨在从低分辨率图像中恢复出高分辨率图像,对遥感图像的高质量解译具有重要意义。针对传统方法依赖多帧图像序列、重建结果过于平滑等问题,提出一种基于边界平衡生成对抗网络的单帧遥感图像超分辨方法。生成器与判别器均设计成带跳跃连接的端到端自编码器结构,为增强生成图像质量及加速网络收敛,使用了一种基于判别器重构误差的损失函数。在NWPU-RESISC45数据集上的实验结果表明,该方法能够提供更多的高频信息,重建结果最接近真实图像,相较于邻近插值和双三次插值方法,PSNR提升约2.70 dB,相较于其他基于深度卷积神经网络的方法,PSNR提升约0.72 dB。
Due to the limitation of imaging devices and transmission conditions, the definition of remote sensing image is difficult to guarantee. Image super-resolution technology aims at recovering high-resolution images from low-resolution images, which is of great significance for high quality interpretation of remote sensing images. In this paper, a single remote sensing image super-resolution method based on boundary equilibrium generative adversarial networks is proposed to solve the problems of traditional methods depending on the multi-frame image sequence and the reconstruction results are too smooth. Both the generator and the discriminator are designed as end to end autoencoder with skip connections.In order to enhance the quality of the generated images and accelerate the convergence of the network, a loss function based on the discriminator reconstruction error is used. The experimental results on the NWPU-RESISC45 dataset show that the proposed method can provide more high frequency information, and the reconstruction results are closest to the real image. Compared with the nearest interpolation and the bicubic interpolation methods, the PSNR is raised about 2.70 dB.Compared with other methods based on the deep convolution neural network, the PSNR is improved by about 0.72 dB.
Due to the limitation of imaging devices and transmission conditions, the definition of remote sensing image is difficult to guarantee. Image super-resolution technology aims at recovering high-resolution images from low-resolution images, which is of great significance for high quality interpretation of remote sensing images. In this paper, a single remote sensing image super-resolution method based on boundary equilibrium generative adversarial networks is proposed to solve the problems of traditional methods depending on the multi-frame image sequence and the reconstruction results are too smooth. Both the generator and the discriminator are designed as end to end autoencoder with skip connections.In order to enhance the quality of the generated images and accelerate the convergence of the network, a loss function based on the discriminator reconstruction error is used. The experimental results on the NWPU-RESISC45 dataset show that the proposed method can provide more high frequency information, and the reconstruction results are closest to the real image. Compared with the nearest interpolation and the bicubic interpolation methods, the PSNR is raised about 2.70 dB.Compared with other methods based on the deep convolution neural network, the PSNR is improved by about 0.72 dB.
引文
[1]Alvarado S T,Fornazari T,Cóstola A,et al.Drivers of fire occurrence in a mountainous Brazilian cerrado savanna:tracking long-term fire regimes using remote sensing[J].Ecological Indicators,2017,78:270-281.
[2]Gong C,Yu S,Joesting H,et al.Determining socioeconomic drivers of urban forest fragmentation with historical remote sensing images[J].Landscape&Urban Planning,2013,117(9):57-65.
[3]He H,Lin Y,Chen F,et al.Inshore ship detection in remote sensing images via weighted pose voting[J].IEEE Transactions on Geoscience&Remote Sensing,2017,55(6):3091-3107.
[4]苏衡,周杰,张志浩.超分辨率图像重建方法综述[J].自动化学报,2013,39(8):1202-1213.
[5]孙旭,李晓光,李嘉锋,等.基于深度学习的图像超分辨率复原研究进展[J].自动化学报,2017,43(5):697-709.
[6]Zheng Tan,Bin Xiangli,Lv Qunbo,et al.A sequence images super-resolution enhancement approach based on frequencydomain[J].Acta Optica Sinica,2017,37(7):0710001.
[7]Xie Q,Yao G,Liu P.Super-resolution reconstruction of satellite video images based on interpolation method[J].Procedia Computer Science,2017,107:454-459.
[8]Panda S S,Prasad M S R S,Jena G.POCS based superresolution image reconstruction using an adaptive regularization parameter[J].International Journal of Computer Science Issues,2011,8(5).
[9]Li F,Xin L,Guo Y,et al.Super-resolution for GaoFen-4remote sensing images[J].IEEE Geoscience&Remote Sensing Letters,2017,PP(99):1-5.
[10]Timofte R,Rothe R,Gool L V.Seven ways to improve example-based single image super resolution[C]//IEEEConference on Computer Vision and Pattern Recognition,2016:1865-1873.
[11]Dong C,Chen C L,He K,et al.Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision,2014:184-199.
[12]Shi W,Caballero J,Huszar F,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//IEEE Conference on Computer Vision and Pattern Recognition,2016:1874-1883.
[13]Kim J,Lee J K,Lee K M.Accurate image super-resolution using very deep convolutional networks[C]//IEEE Conference on Computer Vision and Pattern Recognition,2016:1646-1654.
[14]Mao X J,Shen C,Yang Y B.Image restoration using convolutional auto-encoders with symmetric skip connections[J].arXiv:1606.08921,2016.
[15]Ledig C,Wang Z,Shi W,et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//IEEE Conference on Computer Vision and Pattern Recognition,2017:105-114.
[16]Berthelot D,Schumm T,Metz L.BEGAN:boundary equilibrium generative adversarial networks[J].arXiv:1703.10717,2017.
[17]Goodfellow I J,Pouget-Abadie J,Mirza M,et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems,2014:2672-2680.
[18]Mirza M,Osindero S.Conditional generative adversarial nets[J].arXiv:1411.1784,2014.
[19]Ratliff L J,Burden S A,Sastry S S.Characterization and computation of local Nash equilibria in continuous games[C]//2013 51st Annual Allerton Conference on Communication,Control,and Computing(Allerton),Monticello,IL,2013:917-924.
[20]Badrinarayanan V,Handa A,Cipolla R.SegNet:a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling[J].Computer Science,2015.
[21]Kim J,Lee J K,Lee K M.Deeply-recursive convolutional network for image super-resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition,2016:1637-1645.
[22]Tong T,Li G,Liu X,et al.Image super-resolution using dense skip connections[C]//IEEE International Conference on Computer Vision,2017:4809-4817.
[2]Gong C,Yu S,Joesting H,et al.Determining socioeconomic drivers of urban forest fragmentation with historical remote sensing images[J].Landscape&Urban Planning,2013,117(9):57-65.
[3]He H,Lin Y,Chen F,et al.Inshore ship detection in remote sensing images via weighted pose voting[J].IEEE Transactions on Geoscience&Remote Sensing,2017,55(6):3091-3107.
[4]苏衡,周杰,张志浩.超分辨率图像重建方法综述[J].自动化学报,2013,39(8):1202-1213.
[5]孙旭,李晓光,李嘉锋,等.基于深度学习的图像超分辨率复原研究进展[J].自动化学报,2017,43(5):697-709.
[6]Zheng Tan,Bin Xiangli,Lv Qunbo,et al.A sequence images super-resolution enhancement approach based on frequencydomain[J].Acta Optica Sinica,2017,37(7):0710001.
[7]Xie Q,Yao G,Liu P.Super-resolution reconstruction of satellite video images based on interpolation method[J].Procedia Computer Science,2017,107:454-459.
[8]Panda S S,Prasad M S R S,Jena G.POCS based superresolution image reconstruction using an adaptive regularization parameter[J].International Journal of Computer Science Issues,2011,8(5).
[9]Li F,Xin L,Guo Y,et al.Super-resolution for GaoFen-4remote sensing images[J].IEEE Geoscience&Remote Sensing Letters,2017,PP(99):1-5.
[10]Timofte R,Rothe R,Gool L V.Seven ways to improve example-based single image super resolution[C]//IEEEConference on Computer Vision and Pattern Recognition,2016:1865-1873.
[11]Dong C,Chen C L,He K,et al.Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision,2014:184-199.
[12]Shi W,Caballero J,Huszar F,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//IEEE Conference on Computer Vision and Pattern Recognition,2016:1874-1883.
[13]Kim J,Lee J K,Lee K M.Accurate image super-resolution using very deep convolutional networks[C]//IEEE Conference on Computer Vision and Pattern Recognition,2016:1646-1654.
[14]Mao X J,Shen C,Yang Y B.Image restoration using convolutional auto-encoders with symmetric skip connections[J].arXiv:1606.08921,2016.
[15]Ledig C,Wang Z,Shi W,et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//IEEE Conference on Computer Vision and Pattern Recognition,2017:105-114.
[16]Berthelot D,Schumm T,Metz L.BEGAN:boundary equilibrium generative adversarial networks[J].arXiv:1703.10717,2017.
[17]Goodfellow I J,Pouget-Abadie J,Mirza M,et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems,2014:2672-2680.
[18]Mirza M,Osindero S.Conditional generative adversarial nets[J].arXiv:1411.1784,2014.
[19]Ratliff L J,Burden S A,Sastry S S.Characterization and computation of local Nash equilibria in continuous games[C]//2013 51st Annual Allerton Conference on Communication,Control,and Computing(Allerton),Monticello,IL,2013:917-924.
[20]Badrinarayanan V,Handa A,Cipolla R.SegNet:a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling[J].Computer Science,2015.
[21]Kim J,Lee J K,Lee K M.Deeply-recursive convolutional network for image super-resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition,2016:1637-1645.
[22]Tong T,Li G,Liu X,et al.Image super-resolution using dense skip connections[C]//IEEE International Conference on Computer Vision,2017:4809-4817.