利用图像结构成分的优先块匹配图像修复方法
|
摘要
针对基于样本块图像修复方法中样本块优先级计算易受纹理信息影响,引起修复顺序偏差,进而造成最终修复结果中结构特征不连续的问题,将图像结构成分引入修复过程,提出一种基于图像结构成分计算样本块优先级的图像修复方法.首先通过自适应局部拉普拉斯滤波器对待修复图像进行保边细节平滑处理,得到图像结构成分;然后利用结构成分和原图像共同计算样本块数据项,并以此确定样本块修复优先级,实现图像修复.通过增加结构成分引导方法,在基于等照度线和基于结构张量的图像修复算法上对常用修复测试图像进行实验,结果表明,增加结构成分引导的方法相对于原方法可改进修复效果.
Image inpainting methods base on exemplar have discontinuous structure in the repaired image,which usually caused by the repairing sequence error due to the improper calculation of patches priority.This paper proposes an image inpainting method which calculates the patches priority based on image structural component. First, the adaptive local Laplacian filters are used to edge-aware detail smoothing of the inpainting image, and acquire the structural component image. Then the data term of every border patch is calculated together with the structural component image and the corrupted image, and the priority is computed for each border patch by the data term. Finally, the corrupted image is inpainted according to the obtained priorities. The experimental results show that the inpainting methods guided by the structural component can achieve better visual effect compared to the original methods.
Image inpainting methods base on exemplar have discontinuous structure in the repaired image,which usually caused by the repairing sequence error due to the improper calculation of patches priority.This paper proposes an image inpainting method which calculates the patches priority based on image structural component. First, the adaptive local Laplacian filters are used to edge-aware detail smoothing of the inpainting image, and acquire the structural component image. Then the data term of every border patch is calculated together with the structural component image and the corrupted image, and the priority is computed for each border patch by the data term. Finally, the corrupted image is inpainted according to the obtained priorities. The experimental results show that the inpainting methods guided by the structural component can achieve better visual effect compared to the original methods.
引文
[1]Bertalmio M,Sapiro G,Caselles V,et al.Image inpainting[C]//Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques.New York:ACM Press,2000:417-424
[2]Chan T F,Shen J H.Nontexture inpainting by curvature-driven diffusions[J].Journal of Visual Communication and Image Representation,2001,12(4):436-449
[3]Qin Xujia,Sang Xiansheng,Cheng Shiwei,et al.An improved image inpainting algorithm based on normalized convolution[J].Journal of Computer-Aided Design&Computer Graphics,2011,23(2):371-376(in Chinese)(秦绪佳,桑贤生,程时伟,等.改进的规范化卷积图像修复算法[J].计算机辅助设计与图形学学报,2011,23(2):371-376)
[4]Vinyals O,Toshev A,Bengio S,et al.Show and tell:a neural image caption generator[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:3156-3164
[5]Gatys L A,Ecker A S,Bethge M.Texture synthesis using convolutional neural networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems.Cambridge:MIT Press,2015,1:262-270
[6]Gatys L A,Ecker A S,Bethge M.Image style transfer using convolutional neural networks[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:2414-2423
[7]Goodfellow I J,Pouget-Abadie J,Mirza M,et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.Cambridge:MIT Press,2014,2:2672-2680
[8]Pathak D,Krahenbuhl P,Donahue J,et al.Context encoders:feature learning by inpainting[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:2536-2544
[9]Yang C,Lu X,Lin Z,et al.High-resolution image inpainting using multi-scale neural patch synthesis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:4076-4084
[10]Yeh R A,Chen C,Lim T Y,et al.Semantic image inpainting with deep generative models[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:6882-6890
[11]Iizuka S,Simo-Serra E,Ishikawa H.Globally and locally consistent image completion[J].ACM Transactions on Graphics,2017,36(4):Article No.107
[12]He K M,Sun J.Image completion approaches using the statistics of similar patches[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(12):2423-2435
[13]Komodakis N,Tziritas G.Image completion using efficient belief propagation via priority scheduling and dynamic pruning[J].IEEE Transactions on Image Processing,2007,16(11):2649-2661
[14]Criminisi A,Perez P,Toyama K.Region filling and object removal by exemplar-based image inpainting[J].IEEE Transactions on Image Processing,2004,13(9):1200-1212
[15]Xu Z B,Sun J.Image inpainting by patch propagation using patch sparsity[J].IEEE Transactions on Image Processing,2010,19(5):1153-1165
[16]Li Xufeng,Wang Jing,Liu Hongmin,et al.Image inpainting using feature precedence and patch matching[J].Journal of Computer-Aided Design&Computer Graphics,2016,28(7):1131-1137(in Chinese)(李旭峰,王静,刘红敏,等.特征优先块匹配图像修复算法[J].计算机辅助设计与图形学学报,2016,28(7):1131-1137)
[17]Barnes C,Shechtman E,Finkelstein A,et al.PatchMatch:a randomized correspondence algorithm for structural image editing[J].ACM Transactions on Graphics,2009,28(3):Article No.24
[18]Meur O L,Gautier J,Guillemot C.Examplar-based inpainting based on local geometry[C]//Proceedings of the 18th IEEEConference on Image Processing.Los Alamitos:IEEE Computer Society Press,2011:3401-3404
[19]Qiang Z P,He L B,Chen Y Q,et al.Adaptive fast local Laplacian filters and its edge-aware application[M]//Multimedia Tools and Applications.Heidelberg:Springer,2017:1-21
[20]Bugeau A,Bertalmío M,Caselles V,et al.A comprehensive framework for image inpainting[J].IEEE Transactions on Image Processing,2010,19(10):2634-2645
[21]Meur O L,Guillemot C.Super-resolution-based inpainting[C]//Proceedings of the European Conference on Computer Vision.Heidelberg:Springer,2012:554-567
[22]Korman S,Avidan S.Coherency sensitive hashing[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2011:1607-1614
[23]Paris S,Hasinoff S W,Kautz J.Local Laplacian filters:edgeaware image processing with a Laplacian pyramid[J].ACMTransactions on Graphics,2011,30(4):Article No.68
[24]Aubry M,Paris S,Hasinoff S W,et al.Fast local Laplacian filters:theory and applications[J].ACM Transactions on Graphics,2014,33(5):Article No.167
[2]Chan T F,Shen J H.Nontexture inpainting by curvature-driven diffusions[J].Journal of Visual Communication and Image Representation,2001,12(4):436-449
[3]Qin Xujia,Sang Xiansheng,Cheng Shiwei,et al.An improved image inpainting algorithm based on normalized convolution[J].Journal of Computer-Aided Design&Computer Graphics,2011,23(2):371-376(in Chinese)(秦绪佳,桑贤生,程时伟,等.改进的规范化卷积图像修复算法[J].计算机辅助设计与图形学学报,2011,23(2):371-376)
[4]Vinyals O,Toshev A,Bengio S,et al.Show and tell:a neural image caption generator[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:3156-3164
[5]Gatys L A,Ecker A S,Bethge M.Texture synthesis using convolutional neural networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems.Cambridge:MIT Press,2015,1:262-270
[6]Gatys L A,Ecker A S,Bethge M.Image style transfer using convolutional neural networks[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:2414-2423
[7]Goodfellow I J,Pouget-Abadie J,Mirza M,et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.Cambridge:MIT Press,2014,2:2672-2680
[8]Pathak D,Krahenbuhl P,Donahue J,et al.Context encoders:feature learning by inpainting[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:2536-2544
[9]Yang C,Lu X,Lin Z,et al.High-resolution image inpainting using multi-scale neural patch synthesis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:4076-4084
[10]Yeh R A,Chen C,Lim T Y,et al.Semantic image inpainting with deep generative models[C]//Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:6882-6890
[11]Iizuka S,Simo-Serra E,Ishikawa H.Globally and locally consistent image completion[J].ACM Transactions on Graphics,2017,36(4):Article No.107
[12]He K M,Sun J.Image completion approaches using the statistics of similar patches[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(12):2423-2435
[13]Komodakis N,Tziritas G.Image completion using efficient belief propagation via priority scheduling and dynamic pruning[J].IEEE Transactions on Image Processing,2007,16(11):2649-2661
[14]Criminisi A,Perez P,Toyama K.Region filling and object removal by exemplar-based image inpainting[J].IEEE Transactions on Image Processing,2004,13(9):1200-1212
[15]Xu Z B,Sun J.Image inpainting by patch propagation using patch sparsity[J].IEEE Transactions on Image Processing,2010,19(5):1153-1165
[16]Li Xufeng,Wang Jing,Liu Hongmin,et al.Image inpainting using feature precedence and patch matching[J].Journal of Computer-Aided Design&Computer Graphics,2016,28(7):1131-1137(in Chinese)(李旭峰,王静,刘红敏,等.特征优先块匹配图像修复算法[J].计算机辅助设计与图形学学报,2016,28(7):1131-1137)
[17]Barnes C,Shechtman E,Finkelstein A,et al.PatchMatch:a randomized correspondence algorithm for structural image editing[J].ACM Transactions on Graphics,2009,28(3):Article No.24
[18]Meur O L,Gautier J,Guillemot C.Examplar-based inpainting based on local geometry[C]//Proceedings of the 18th IEEEConference on Image Processing.Los Alamitos:IEEE Computer Society Press,2011:3401-3404
[19]Qiang Z P,He L B,Chen Y Q,et al.Adaptive fast local Laplacian filters and its edge-aware application[M]//Multimedia Tools and Applications.Heidelberg:Springer,2017:1-21
[20]Bugeau A,Bertalmío M,Caselles V,et al.A comprehensive framework for image inpainting[J].IEEE Transactions on Image Processing,2010,19(10):2634-2645
[21]Meur O L,Guillemot C.Super-resolution-based inpainting[C]//Proceedings of the European Conference on Computer Vision.Heidelberg:Springer,2012:554-567
[22]Korman S,Avidan S.Coherency sensitive hashing[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2011:1607-1614
[23]Paris S,Hasinoff S W,Kautz J.Local Laplacian filters:edgeaware image processing with a Laplacian pyramid[J].ACMTransactions on Graphics,2011,30(4):Article No.68
[24]Aubry M,Paris S,Hasinoff S W,et al.Fast local Laplacian filters:theory and applications[J].ACM Transactions on Graphics,2014,33(5):Article No.167