重彩画的风格转移算法
|
摘要
如何在实现人物风格转移的同时保持人物形体结构的协调性和脸部辨识的一致性,是目前人物画风格转移较难解决的问题.为此,将云南重彩画作为风格转移目标,把输入照片转移成目标风格,提出人物绘画作品和纹理精细绘画作品的风格转移算法.针对重彩画人物肢体修长的特点提出人体图像重塑,对照片人物进行变形;为保留人物形体结构和重彩画精细纹理,提出按语义的风格转移和图像类比相结合的解决方案,增加结构保持正则项;针对重彩画线条分明灵动的特点,提出对风格转移图进行线条感增强,结合L_0梯度最小化平滑图像、各向异性的DoG(difference-of-Gaussians)滤波器和线条图缺口检测以及补全算法提取出光滑连贯的线条.运用该算法对重彩画风格进行转移,转移结果表现出重彩画人物肢体修长的美感、精细的纹理特征和线条分明灵动的特点.
How to achieve the style transfer of figure paintings and maintain the harmony structure of body and consistency of face recognition at the same time, is a difficult problem in the style transfer of the figure paintings at present. For this purpose, the Yunnan heavy color painting is chosen as the target style to carry out the research of the style transfer of paintings with fine texture and figure paintings. Aimed at the aesthetic feeling of slender body of heavy color painting, the human body image restoring is put forward to reshaping the character in the image. To keep body structure and fine texture of the heavy color paintings, the technique of combining style transfer based on semantics and image analogy is proposed, and the regularization term of structure maintaining is added into loss function. For the distinct line in the heavy color painting, the image smoothing method via L_0 gradient minimization, the anisotropic DoG(difference of Gaussians) filter and the gap detection are combined to extract the smooth and coherent line drawing from content images to enhance the line sense. This paper uses this algorithm to transfer the style of heavy color painting, the result shows the aesthetic of character body slender and the fine texture features and the sharp lines of heavy color painting.
How to achieve the style transfer of figure paintings and maintain the harmony structure of body and consistency of face recognition at the same time, is a difficult problem in the style transfer of the figure paintings at present. For this purpose, the Yunnan heavy color painting is chosen as the target style to carry out the research of the style transfer of paintings with fine texture and figure paintings. Aimed at the aesthetic feeling of slender body of heavy color painting, the human body image restoring is put forward to reshaping the character in the image. To keep body structure and fine texture of the heavy color paintings, the technique of combining style transfer based on semantics and image analogy is proposed, and the regularization term of structure maintaining is added into loss function. For the distinct line in the heavy color painting, the image smoothing method via L_0 gradient minimization, the anisotropic DoG(difference of Gaussians) filter and the gap detection are combined to extract the smooth and coherent line drawing from content images to enhance the line sense. This paper uses this algorithm to transfer the style of heavy color painting, the result shows the aesthetic of character body slender and the fine texture features and the sharp lines of heavy color painting.
引文
[1]Guo Y W,Yu J H,Xu X D,et al.Example based painting generation[J].Journal of Zhejiang University Science A,2006,7(7):1152-1159
[2]Yan Li,Pu Yuanyuan,Zhang Mengchen,et al.Content guided pencil drawing style[J].Journal of Computer-Aided Design&Computer Graphics,2017,29(7):1292-1302(in Chinese)(闫莉,普园媛,张梦晨,等.绘制内容指导的铅笔画风格实现[J].计算机辅助设计与图形学学报,2017,29(7):1292-1302)
[3]Zhao M T,Zhu S C.Portrait painting using active templates[C]//Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and Rendering.Aire-la-Ville:Eurographics Association Press,2011:117-124
[4]Zhao Y D,Jin X G,Xu Y Q,et al.Parallel style-aware image cloning for artworks[J].IEEE Transactions on Visualization and Computer Graphics,2015,21(2):229-240
[5]Winnem?ller H,Olsen S C,Gooch B.Real-time video abstraction[J].ACM Transactions on Graphics,2006,25(3):1221-1226
[6]Chen X W,Jin X,Zhao Q P,et al.Artistic illumination transfer for portraits[J].Computer Graphics Forum,2012,31(4):1425-1434
[7]Reinhard E,Ashikhmin M,Gooch B,et al.Color transfer between images[J].IEEE Computer Graphics and Applications,2002,21(5):34-41
[8]Gatys L A,Ecker A S,Bethge M.Image style transfer using convolutional neural networks[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:2414-2423
[9]Wang X,Oxholm G,Zhang D,et al.Multimodal transfer:a hierarchical deep convolutional neural network for fast artistic style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2017:7178-7186
[10]Johnson J,Alahi A,Li F F.Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of the European Conference on Computer Vision.Heidelberg:Springer,2016,9906:694-711
[11]Ulyanov D,Lebedev V,Vedaldi A,et al.Texture networks:feed-forward synthesis of textures and stylized images[OL].[2018-06-29].http://cn.arxiv.org/abs/1603.03417
[12]Dumoulin V,Shlens J,Kudlur M.A learned representation for artistic style[OL].[2018-06-29].http://cn.arxiv.org/abs/1610.07629
[13]Chen T Q,Schmidt M.Fast patch-based style transfer of arbitrary style[OL].[2018-06-29].http://cn.arxiv.org/abs/1612.04337
[14]Chen D D,Yuan L,Liao J,et al.StyleBank:an explicit representation for neural image style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:2770-2779
[15]Risser E,Wilmot P,Barnes C.Stable and controllable neural texture synthesis and style transfer using histogram losses[OL].[2018-06-29].http://cn.arxiv.org/abs/1701.08893
[16]Gatys L A,Bethge M,Hertzmann A,et al.Preserving color in neural artistic style transfer[OL].[2018-06-29].http://cn.arxiv.org/abs/1606.05897
[17]Gatys L A,Ecker A S,Bethge M,et al.Controlling perceptual factors in neural style transfer[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:3730-3738
[18]Selim A,Elgharib M,Doyle L.Painting style transfer for head portraits using convolutional neural networks[J].ACM Transactions on Graphics,2016,35(4):Article No.129
[19]Liu S,Ou X Y,Qian R H,et al.Makeup like a superstar:deep localized makeup transfer network[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence.Palo Alto:AAAI Press,2016:2568-2575
[20]Shih Y C,Paris S,Barnes C,et al.Style transfer for headshot portraits[J].ACM Transactions on Graphics,2014,33(4):Article No.148
[21]Luan F J,Paris S,Shechtman E,et al.Deep photo style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:6997-7005
[22]Zhu J Y,Park T,Isola P,et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:2242-2251
[23]Kim T,Cha M,Kim H,et al.Learning to discover cross-domain relations with generative adversarial networks[OL].[2018-06-29].http://cn.arxiv.org/abs/1703.05192
[24]Yi Z L,Zhang H,Tan P,et al.DualGAN:unsupervised dual learning for image-to-image translation[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:2868-2876
[25]Liao J,Yao Y,Yuan L,et al.Visual attribute transfer through deep image analogy[J].ACM Transactions on Graphics,2017,36(4):Article No.120
[26]Pu Yuanyuan.Research on digital simulation and synthesis of Yunnan heavy color painting[D].Kunming:Yunnan University.School of Information Science and Engineering,2010(in Chinese)(普园媛.云南重彩画艺术风格的数字模拟及合成技术研究[D].昆明:云南大学信息学院,2010)
[27]Lu Liwen,Pu Yuanyuan,Liu Yuqing,et al.A portrait generation algorithm of Yunnan heavy color painting[J].Journal of Graphics,2013,34(3):126-133(in Chinese)(卢丽稳,普园媛,刘玉清,等.云南重彩画人脸肖像生成算法[J].图学学报,2013,34(3):126-133)
[28]Xu L,Lu C W,Xu Y,et al.Image smoothing via L0 gradient minimization[J].ACM Transactions on Graphics,2011,30(6):Article No.174
[29]Kang H,Lee S,Chui C K.Coherent line drawing[C]//Proceedings of the 5th International Symposium on Non-Photorealistic Animation and Rendering.New York:ACM Press,2007:43-50
[30]Sasaki K,Iizuka S,Simo-Serra E,et al.Joint gap detection and inpainting of line drawings[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:5768-5776
[31]Criminisi A,Perez P,Toyama K.Region filling and object removal by exemplar-based image inpainting[J].IEEE Transactions on Image Processing,2004,13(9):1200-1212
[32]Schaefer S,Mcphail T,Warren J.Image deformation using moving least squares[C]//Computer Graphics Proceedings,Annual Conference Series,ACM SIGGRAPH.New York:ACM Press,2006:533-540
[33]Igarashi T,Moscovich T,Hughes J F.As-rigid-as-possible shape manipulation[J].ACM Transactions on Graphics,2005,24(3):1134-1141
[34]Zhou S Z,Fu H B,Liu L G,et al.Parametric reshaping of human bodies in images[J].ACM Transactions on Graphics,2010,29(4):Article No.126
[35]Hertzmann A,Jacobs C E,Oliver N,et al.Image Analogies[C].//Computer Graphics Proceedings,Annual Conference Series,ACM SIGGRAPH.New York:ACM Press,2001:327-340
[36]Rez P,Gangnet M,Blake A.Poisson image editing[J].ACMTransactions on Graphics,2003,22(3):313-318
[2]Yan Li,Pu Yuanyuan,Zhang Mengchen,et al.Content guided pencil drawing style[J].Journal of Computer-Aided Design&Computer Graphics,2017,29(7):1292-1302(in Chinese)(闫莉,普园媛,张梦晨,等.绘制内容指导的铅笔画风格实现[J].计算机辅助设计与图形学学报,2017,29(7):1292-1302)
[3]Zhao M T,Zhu S C.Portrait painting using active templates[C]//Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and Rendering.Aire-la-Ville:Eurographics Association Press,2011:117-124
[4]Zhao Y D,Jin X G,Xu Y Q,et al.Parallel style-aware image cloning for artworks[J].IEEE Transactions on Visualization and Computer Graphics,2015,21(2):229-240
[5]Winnem?ller H,Olsen S C,Gooch B.Real-time video abstraction[J].ACM Transactions on Graphics,2006,25(3):1221-1226
[6]Chen X W,Jin X,Zhao Q P,et al.Artistic illumination transfer for portraits[J].Computer Graphics Forum,2012,31(4):1425-1434
[7]Reinhard E,Ashikhmin M,Gooch B,et al.Color transfer between images[J].IEEE Computer Graphics and Applications,2002,21(5):34-41
[8]Gatys L A,Ecker A S,Bethge M.Image style transfer using convolutional neural networks[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:2414-2423
[9]Wang X,Oxholm G,Zhang D,et al.Multimodal transfer:a hierarchical deep convolutional neural network for fast artistic style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2017:7178-7186
[10]Johnson J,Alahi A,Li F F.Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of the European Conference on Computer Vision.Heidelberg:Springer,2016,9906:694-711
[11]Ulyanov D,Lebedev V,Vedaldi A,et al.Texture networks:feed-forward synthesis of textures and stylized images[OL].[2018-06-29].http://cn.arxiv.org/abs/1603.03417
[12]Dumoulin V,Shlens J,Kudlur M.A learned representation for artistic style[OL].[2018-06-29].http://cn.arxiv.org/abs/1610.07629
[13]Chen T Q,Schmidt M.Fast patch-based style transfer of arbitrary style[OL].[2018-06-29].http://cn.arxiv.org/abs/1612.04337
[14]Chen D D,Yuan L,Liao J,et al.StyleBank:an explicit representation for neural image style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:2770-2779
[15]Risser E,Wilmot P,Barnes C.Stable and controllable neural texture synthesis and style transfer using histogram losses[OL].[2018-06-29].http://cn.arxiv.org/abs/1701.08893
[16]Gatys L A,Bethge M,Hertzmann A,et al.Preserving color in neural artistic style transfer[OL].[2018-06-29].http://cn.arxiv.org/abs/1606.05897
[17]Gatys L A,Ecker A S,Bethge M,et al.Controlling perceptual factors in neural style transfer[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:3730-3738
[18]Selim A,Elgharib M,Doyle L.Painting style transfer for head portraits using convolutional neural networks[J].ACM Transactions on Graphics,2016,35(4):Article No.129
[19]Liu S,Ou X Y,Qian R H,et al.Makeup like a superstar:deep localized makeup transfer network[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence.Palo Alto:AAAI Press,2016:2568-2575
[20]Shih Y C,Paris S,Barnes C,et al.Style transfer for headshot portraits[J].ACM Transactions on Graphics,2014,33(4):Article No.148
[21]Luan F J,Paris S,Shechtman E,et al.Deep photo style transfer[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:6997-7005
[22]Zhu J Y,Park T,Isola P,et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:2242-2251
[23]Kim T,Cha M,Kim H,et al.Learning to discover cross-domain relations with generative adversarial networks[OL].[2018-06-29].http://cn.arxiv.org/abs/1703.05192
[24]Yi Z L,Zhang H,Tan P,et al.DualGAN:unsupervised dual learning for image-to-image translation[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2017:2868-2876
[25]Liao J,Yao Y,Yuan L,et al.Visual attribute transfer through deep image analogy[J].ACM Transactions on Graphics,2017,36(4):Article No.120
[26]Pu Yuanyuan.Research on digital simulation and synthesis of Yunnan heavy color painting[D].Kunming:Yunnan University.School of Information Science and Engineering,2010(in Chinese)(普园媛.云南重彩画艺术风格的数字模拟及合成技术研究[D].昆明:云南大学信息学院,2010)
[27]Lu Liwen,Pu Yuanyuan,Liu Yuqing,et al.A portrait generation algorithm of Yunnan heavy color painting[J].Journal of Graphics,2013,34(3):126-133(in Chinese)(卢丽稳,普园媛,刘玉清,等.云南重彩画人脸肖像生成算法[J].图学学报,2013,34(3):126-133)
[28]Xu L,Lu C W,Xu Y,et al.Image smoothing via L0 gradient minimization[J].ACM Transactions on Graphics,2011,30(6):Article No.174
[29]Kang H,Lee S,Chui C K.Coherent line drawing[C]//Proceedings of the 5th International Symposium on Non-Photorealistic Animation and Rendering.New York:ACM Press,2007:43-50
[30]Sasaki K,Iizuka S,Simo-Serra E,et al.Joint gap detection and inpainting of line drawings[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:5768-5776
[31]Criminisi A,Perez P,Toyama K.Region filling and object removal by exemplar-based image inpainting[J].IEEE Transactions on Image Processing,2004,13(9):1200-1212
[32]Schaefer S,Mcphail T,Warren J.Image deformation using moving least squares[C]//Computer Graphics Proceedings,Annual Conference Series,ACM SIGGRAPH.New York:ACM Press,2006:533-540
[33]Igarashi T,Moscovich T,Hughes J F.As-rigid-as-possible shape manipulation[J].ACM Transactions on Graphics,2005,24(3):1134-1141
[34]Zhou S Z,Fu H B,Liu L G,et al.Parametric reshaping of human bodies in images[J].ACM Transactions on Graphics,2010,29(4):Article No.126
[35]Hertzmann A,Jacobs C E,Oliver N,et al.Image Analogies[C].//Computer Graphics Proceedings,Annual Conference Series,ACM SIGGRAPH.New York:ACM Press,2001:327-340
[36]Rez P,Gangnet M,Blake A.Poisson image editing[J].ACMTransactions on Graphics,2003,22(3):313-318