基于生成对抗网络的图像超分辨率方法
|
摘要
为了解决生成对抗网络(Generative adversarial network, GAN)训练不稳定问题,降低模型复杂度,加快网络学习速率,提高超分辨率图像的视觉效果和重建速率,提出了一种基于改进生成对抗网络的图像超分辨率方法。该方法以改进的生成对抗网络为模型,通过粗粒度主体内容和细粒度细节边缘结合的方式提取图像特征,利用线性组合的方式重建超分辨率图像,采用Wasserstein距离优化生成对抗网络。实验结果表明:该方法能够生成视觉效果良好的超分辨率图像,在Set5、Set14等测试集上,其主观视觉评价和客观量化指标(PSNR、SSIM)都优于SRGAN方法。该方法通过重新设计网络模型,使得特征提取更为全面,网络训练更加充分,有助于提高超分辨率图像重建速度,提高图像质量。
To solve the problem of training instability of generative adversarial network, reduce model complexity, and speed up network learning rate, and improve the visual effect and reconstruction rate of super-resolution image, an image super-resolution method based on improved generative adversarial networks is proposed. In the method, improved generative adversarial network is taken as the model, image features are extracted by combining main content of coarse granularity with detail edge of fine granularity, super-resolution images are reconstructed by means of linear combination, and generative adversarial network is optimized via Wasserstein distance. Experimental results show that super-resolution images with advanced visual effect can be generated with this method, and the method is superior to SRGAN in respect of subjective evaluation and objective quantification(PSNR/SSIM) in Set5, Set14 and such test sets. With this method, by redesigning the network model, feature extraction is conducted more comprehensively, and network training is conducted more completely, which helps to improve the speed of super-resolution image reconstruction and image quality.
To solve the problem of training instability of generative adversarial network, reduce model complexity, and speed up network learning rate, and improve the visual effect and reconstruction rate of super-resolution image, an image super-resolution method based on improved generative adversarial networks is proposed. In the method, improved generative adversarial network is taken as the model, image features are extracted by combining main content of coarse granularity with detail edge of fine granularity, super-resolution images are reconstructed by means of linear combination, and generative adversarial network is optimized via Wasserstein distance. Experimental results show that super-resolution images with advanced visual effect can be generated with this method, and the method is superior to SRGAN in respect of subjective evaluation and objective quantification(PSNR/SSIM) in Set5, Set14 and such test sets. With this method, by redesigning the network model, feature extraction is conducted more comprehensively, and network training is conducted more completely, which helps to improve the speed of super-resolution image reconstruction and image quality.
引文
[1] Zhou F,Yang W,Liao Q.Interpolation-based image super-resolution using multisurface fitting[J].IEEE Transactions on Image Processing,2012,21(7):3312-3318.
[2] 曾凯,丁世飞.图像超分辨率重建的研究进展[J].计算机工程与应用,2017,53(16):29-35.
[3] 胡长胜,詹曙,吴从中.基于深度特征学习的图像超分辨率重建[J].自动化学报,2017,43(5):814-821.
[4] Dong C,Loy C C,He K,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[5] Kim J,Lee J K,Lee K M.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas.IEEE,2016:1637-1645.
[6] Kim J,Lee J K,Lee K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas.IEEE,2016:1646-1654.
[7] Tai Y,Yang J,Liu X.Image super-resolution via deep recursive residual network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Hawaii.IEEE,2017:2790-2798.
[8] Lai W S,Huang J B,Ahuja N,et al.Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition,Hawaii.IEEE,2017:5835-5843.
[9] Ledig C,Theis L,Huszár F,et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//IEEE Conference on Computer Vision and Pattern Recognition,Hawaii.IEEE,2017:105-114.
[10] 孙旭,李晓光,李嘉锋,等.基于深度学习的图像超分辨率复原研究进展[J].自动化学报,2017,43(5):697-709.
[11] Goodfellow I,Pouget-Abadie J,Mirza M,et al.Generative adversarial nets[C]//Advances in Neural Information Processing Systems.Cambridge:MIT Press,2014:2672-2680.
[12] 王一宁,秦品乐,李传朋,等.基于残差神经网络的图像超分辨率改进算法[J].计算机应用,2018,38(1):246-254.
[13] Pan J,Liu S,Sun D,et al.Learning dual convolutional neural networks for low-level vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Salt Lake City.IEEE,2018:3070-3079.
[14] 麻旋,戴曙光.基于残差网络的图像超分辨率算法改进研究[J].软件导刊,2018,17(4):91-93.
[15] 王耀杰,钮可,杨晓元.基于生成对抗网络的信息隐藏方案[J].计算机应用,2018,38(10),2923-2928.
[16] Arjovsky M,Chintala S,Bottou L.Wasserstein generative adversarial networks[C]//International Conference on Machine Learning.Sydney:PMLR,2017:214-223.
[17] 杨延涛.L_p空间中Lipschitz强单调算子方程解的迭代算法[J].浙江大学学报(理学版),2018,45(4):405-408.
[18] Andrychowicz M,Denil M,Gomez S,et al.Learning to learn by gradient descent by gradient descent[C]//Advances in Neural Information Processing Systems.Cambridge:MIT Press,2016:3981-3989.
[19] Hore A,Ziou D.Image quality metrics:PSNR vs.SSIM[C]// International Conference on Pattern Recognition,Istanbul.IEEE,2010:2366-2369.
[2] 曾凯,丁世飞.图像超分辨率重建的研究进展[J].计算机工程与应用,2017,53(16):29-35.
[3] 胡长胜,詹曙,吴从中.基于深度特征学习的图像超分辨率重建[J].自动化学报,2017,43(5):814-821.
[4] Dong C,Loy C C,He K,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[5] Kim J,Lee J K,Lee K M.Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas.IEEE,2016:1637-1645.
[6] Kim J,Lee J K,Lee K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas.IEEE,2016:1646-1654.
[7] Tai Y,Yang J,Liu X.Image super-resolution via deep recursive residual network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Hawaii.IEEE,2017:2790-2798.
[8] Lai W S,Huang J B,Ahuja N,et al.Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition,Hawaii.IEEE,2017:5835-5843.
[9] Ledig C,Theis L,Huszár F,et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//IEEE Conference on Computer Vision and Pattern Recognition,Hawaii.IEEE,2017:105-114.
[10] 孙旭,李晓光,李嘉锋,等.基于深度学习的图像超分辨率复原研究进展[J].自动化学报,2017,43(5):697-709.
[11] Goodfellow I,Pouget-Abadie J,Mirza M,et al.Generative adversarial nets[C]//Advances in Neural Information Processing Systems.Cambridge:MIT Press,2014:2672-2680.
[12] 王一宁,秦品乐,李传朋,等.基于残差神经网络的图像超分辨率改进算法[J].计算机应用,2018,38(1):246-254.
[13] Pan J,Liu S,Sun D,et al.Learning dual convolutional neural networks for low-level vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Salt Lake City.IEEE,2018:3070-3079.
[14] 麻旋,戴曙光.基于残差网络的图像超分辨率算法改进研究[J].软件导刊,2018,17(4):91-93.
[15] 王耀杰,钮可,杨晓元.基于生成对抗网络的信息隐藏方案[J].计算机应用,2018,38(10),2923-2928.
[16] Arjovsky M,Chintala S,Bottou L.Wasserstein generative adversarial networks[C]//International Conference on Machine Learning.Sydney:PMLR,2017:214-223.
[17] 杨延涛.L_p空间中Lipschitz强单调算子方程解的迭代算法[J].浙江大学学报(理学版),2018,45(4):405-408.
[18] Andrychowicz M,Denil M,Gomez S,et al.Learning to learn by gradient descent by gradient descent[C]//Advances in Neural Information Processing Systems.Cambridge:MIT Press,2016:3981-3989.
[19] Hore A,Ziou D.Image quality metrics:PSNR vs.SSIM[C]// International Conference on Pattern Recognition,Istanbul.IEEE,2010:2366-2369.