基于局部均方差的神经网络图像风格转换
|
摘要
Gatys等人首次采用基于深度学习的方法,将图像的内容与风格进行分离与重组,使图像可以进行任意的风格转换,至此开创一个新的领域,即基于神经网络的图像风格化转换。该文在Gatys等人的研究基础上,引入局部均方差去噪方法,将局部均方差作为神经网络损失函数的一部分,同时结合内容损失函数与风格函数,将此三种损失函数的加权代数和作为神经网络的总损失函数。结果表明,该文方法在进行图像风格转换时,有效提升了风格转换算法输出的图像质量,使得图像噪声点明显减少,图像更加平滑。
Gatys and others first use the deep learning-based method to separate and reorganize the contents and styles of images,so that image style can be transformed arbitrarily,which opens up a new field of image style transformation based on neural networks. On the basis of the research of Gatys and others,the local mean square error denoising method is introduced in this paper. The local mean square error is taken as part of the neural network loss function,and the weighted algebraic sum of the three loss functions is taken as the total loss function of the neural network by combining the content loss function and style function. The results show that the method proposed in this paper can effectively improve the image quality output by the style transformation algorithm while performing image style transformation,which makes image noise points significantly reduced and produces smoother images.
Gatys and others first use the deep learning-based method to separate and reorganize the contents and styles of images,so that image style can be transformed arbitrarily,which opens up a new field of image style transformation based on neural networks. On the basis of the research of Gatys and others,the local mean square error denoising method is introduced in this paper. The local mean square error is taken as part of the neural network loss function,and the weighted algebraic sum of the three loss functions is taken as the total loss function of the neural network by combining the content loss function and style function. The results show that the method proposed in this paper can effectively improve the image quality output by the style transformation algorithm while performing image style transformation,which makes image noise points significantly reduced and produces smoother images.
引文
[1] HAEBERLI P. Paint by numbers:abstract image representations[J]. ACM SIGGRAPH computer graphics,1990,24(4):207-214.
[2] HERTZMANN A. Tutorial:a survey of stroke-based rendering[J]. IEEE computer graphics and applications,2003,23(4):70-81.
[3] KANG D,KONG P,YOON K,et al. Directional texture transfer for video[J]. Multimedia tools&applications,2015,74(1):245-258.
[4] GATYS L A,ECKER A S,BETHGE M. Image style transfer using convolutional neural networks[C]//Proceedings of 2016IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:2414-2423.
[5] EFROS A A,LEUNG T K. Texture synthesis by non-parametric sampling[C]//Proceedings of IEEE International Conference on Computer Vision. Corfu:IEEE,1999:1033-1038.
[6] WEI L Y,LEVOY M. Fast texture synthesis using tree-structured vector quantization[C]//Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques.New York:ACM,2000:479-488.
[7] EFROS A A,FREEMAN W T. Image quilting for texture synthesisand transfer[C]//Proceedingsofthe 28th AnnualConference on Computer Graphics and Interactive Techniques. New York:ACM,2001:341-346.
[8] SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL].[2015-04-10]https://arxiv.org/pdf/1409.1556v6.pdf.
[9] RUSSAKOVSKY O,DENG J,SU H,et al. ImageNet large scale visual recognition challenge[J]. International journal of computer vision,2015,115(3):211-252.
[10] GüCLüU,GERVEN M A J. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream[J]. Journal of neuroscience,2015,35(27):10005-10014.
[11] GATYS L A,ECKER A S,BETHGE M. Texture synthesis using convolutional neural networks[J/OL].[2015-11-06].https://arxiv.org/pdf/1505.07376v3.pdf.
[12]张航,曹瞻.基于局部均值与方差的图像中值滤波方法[J].中南大学学报(自然科学版),2013,44(z2):381-384.ZHANG Hang,CAO Zhan. Image median filter method based on local mean and variance[J]. Journal of Central South University(Science and technology),2013,44(S2):381-384.
[2] HERTZMANN A. Tutorial:a survey of stroke-based rendering[J]. IEEE computer graphics and applications,2003,23(4):70-81.
[3] KANG D,KONG P,YOON K,et al. Directional texture transfer for video[J]. Multimedia tools&applications,2015,74(1):245-258.
[4] GATYS L A,ECKER A S,BETHGE M. Image style transfer using convolutional neural networks[C]//Proceedings of 2016IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:2414-2423.
[5] EFROS A A,LEUNG T K. Texture synthesis by non-parametric sampling[C]//Proceedings of IEEE International Conference on Computer Vision. Corfu:IEEE,1999:1033-1038.
[6] WEI L Y,LEVOY M. Fast texture synthesis using tree-structured vector quantization[C]//Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques.New York:ACM,2000:479-488.
[7] EFROS A A,FREEMAN W T. Image quilting for texture synthesisand transfer[C]//Proceedingsofthe 28th AnnualConference on Computer Graphics and Interactive Techniques. New York:ACM,2001:341-346.
[8] SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL].[2015-04-10]https://arxiv.org/pdf/1409.1556v6.pdf.
[9] RUSSAKOVSKY O,DENG J,SU H,et al. ImageNet large scale visual recognition challenge[J]. International journal of computer vision,2015,115(3):211-252.
[10] GüCLüU,GERVEN M A J. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream[J]. Journal of neuroscience,2015,35(27):10005-10014.
[11] GATYS L A,ECKER A S,BETHGE M. Texture synthesis using convolutional neural networks[J/OL].[2015-11-06].https://arxiv.org/pdf/1505.07376v3.pdf.
[12]张航,曹瞻.基于局部均值与方差的图像中值滤波方法[J].中南大学学报(自然科学版),2013,44(z2):381-384.ZHANG Hang,CAO Zhan. Image median filter method based on local mean and variance[J]. Journal of Central South University(Science and technology),2013,44(S2):381-384.