图像修补技术的研究
|
摘要
数字图像修补问题是图像处理中的一个热点问题,主要是利用一定的数学模型针对有缺损的图像进行修复,或者从图像中去除指定的目标和文字,以达到特定的目的。
本文首先介绍图像修补的研究背景、意义,以及现状;重点讨论了对于小块的缺损有良好修补效果的基于偏微分方程(PDE)的修补模型,包括BSCB模型,TV模型,Mumford-Shah-Euler模型和CDD模型;接着详细分析了Split Bregman迭代方法,介绍了两种Split Bregman迭代方法的运用。
CDD模型是通过引入曲率项求解高阶偏微分方程来进行图像修补,在求解偏微分方程过程中,大量迭代运算导致修复速度非常缓慢。另一方面,Split Bregman方法具有迭代次数少、运算速度快和适于解决L1正则化问题等特点。本文把Split Bregman方法引入到CDD图像修补模型中,提出了一种基于Split Bregman迭代的CDD图像修补方法,取得了很好的效果。
最后,介绍了适合于大块缺损的纹理合成的图像修补方法。重点研究了非参数性采样的纹理合成和基于优先权的纹理合成方法。
Digital image inpainting,which is a hot issue in the field of image processing,is mainly used to inpaint damaged images or remove of objects or words in images with some mathematical models to achieve special goals.
In this thesis, the background of the research, the significance and the present situation of image inpainting are firstly introduced. Then, we focus on some of PDE-based image inpainting models: BSCB model,TV inpainting, Mumford-Shah-Euler inpainting, CDD model,which have good inpainting effects for those damaged in some small blocks. Thereafter, we analyse Split Bregman method in detail and present its two applications.
For using the curvature term in the process of image inpainting with CDD model, a large number of iterations to solve the high order PDEgreatly lowers the speed of inpainting.To speed up the algorithm, a Split Bregman method ,which is operated faster and good for solving L1 regularization problems, is introduced to the CDD model. The experiments show that our new algorithm is much faster and with better vision effects.
Finally,this thesis introduces the inpainting models based on texture synthesis,which have good inpainting effect on large damaged blocks. Texture synthesis methods by non-parametric sampling and based on priority are studied respectively.
本文首先介绍图像修补的研究背景、意义,以及现状;重点讨论了对于小块的缺损有良好修补效果的基于偏微分方程(PDE)的修补模型,包括BSCB模型,TV模型,Mumford-Shah-Euler模型和CDD模型;接着详细分析了Split Bregman迭代方法,介绍了两种Split Bregman迭代方法的运用。
CDD模型是通过引入曲率项求解高阶偏微分方程来进行图像修补,在求解偏微分方程过程中,大量迭代运算导致修复速度非常缓慢。另一方面,Split Bregman方法具有迭代次数少、运算速度快和适于解决L1正则化问题等特点。本文把Split Bregman方法引入到CDD图像修补模型中,提出了一种基于Split Bregman迭代的CDD图像修补方法,取得了很好的效果。
最后,介绍了适合于大块缺损的纹理合成的图像修补方法。重点研究了非参数性采样的纹理合成和基于优先权的纹理合成方法。
Digital image inpainting,which is a hot issue in the field of image processing,is mainly used to inpaint damaged images or remove of objects or words in images with some mathematical models to achieve special goals.
In this thesis, the background of the research, the significance and the present situation of image inpainting are firstly introduced. Then, we focus on some of PDE-based image inpainting models: BSCB model,TV inpainting, Mumford-Shah-Euler inpainting, CDD model,which have good inpainting effects for those damaged in some small blocks. Thereafter, we analyse Split Bregman method in detail and present its two applications.
For using the curvature term in the process of image inpainting with CDD model, a large number of iterations to solve the high order PDEgreatly lowers the speed of inpainting.To speed up the algorithm, a Split Bregman method ,which is operated faster and good for solving L1 regularization problems, is introduced to the CDD model. The experiments show that our new algorithm is much faster and with better vision effects.
Finally,this thesis introduces the inpainting models based on texture synthesis,which have good inpainting effect on large damaged blocks. Texture synthesis methods by non-parametric sampling and based on priority are studied respectively.
引文
[1] M. Bertalmio, G. Sapiro, V. Caselles, C. Ballester, Image Inpainting [A].In: Proceedings International Conference on Computer Graphics and Interactive Techniques[c],New Orleans Louisiana,USA ,2000:417-424.
[2] Tony F. Chan, Jianhong Shen.Mathematical Models for Local Non-Texture Inpainting [J].SIAM Journal on Applied Mathematics, 2002, 62:1019-1043.
[3] Tony F. Chan, Jianhong Shen.Non-Texture Inpainting by Curvature-Driven Diffusions (CDD) [R].Technical Report CAM 00-35, Image processing Research Group, UCLA, 2000.
[4] Tsai A. Yezzi J A. Willsky A S. Curve evolution implantation of the Mumford function for image segmentation, denoising, interpolation and magnification[J]. IEEE Transactions on Image Processing, 2001, 10(8): 1169-1186
[5] Chan T F, Kang S H, Shen J H, Euler’s Elastica and Curvature Based Inpainting [J]. SIAM Journal of Applied Mathematics, 2002, 63(2):564-592.
[6] Esedoglu S, Shen J H.Digital Inpainting Based on the Mumford-Shah-Euler Image Model [J]. European Journal on Applied Mathematics, 2002, 13(4):353-370.
[7] Harald Grossauer A Combined PDE and texture Synthesis Approach to Inpainting.
[8]张红英,彭启琮《数字图像修复技术综述》,中国图像图形学报,2007年1月,第12卷第1期
[9]庄建飞,《基于偏微分方程(PDE)的图像修补》学位论文南京理工大学2006-06-01
[10] C.Ballester,M.Bertalmiol,V.Caselles,G.Sapiro.andJ.Vergera.Filling-in by joint interpolation of vector fileds and Gray levels.IEEE Tran.Image Processing,10(8):1200-1211,2001
[11]郑孟琦《图像修补模型、算法及其应用研究》,学位论文,南京理工大学,2005
[13] Chan T F. Shen J H. Non-texture inpainting by curvature-driven diffusions(CDD)[J].Journal of Visual Communication and Image Representation,2001,12(4):436~449.
[14] Rudin I., Osher S, Faterni E. Nonlinear total variation based noise removal algorithms[J]. Physica D, 1992. 60(1~4):259~268
[15] T.F.Chan and J.Shen.Mathematical models for local deterministic inpaintings.SIAM J,Appl.Math,62(3):1019-1043,2001
[16] D.Mumford and J.Shah. Optimal approximations by piecewise smooth functions and associated variational problems. Comm.Pure Applied. Math. 79(12),1988
[17] D.Mumford. Elastica and computer vision. In C. L. Bajaj,editor, Algebraic Geometriy and its Applications, pages 491-506.Springer-Verlag, New York,1994
[18] L Bregman . The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex optimization[J]. USSR Computational Mathematical Physics,1967, 7:200~217
[19] Stanley Osher, Martin Burger, Donalb, Jinjun Xu, Wotao Yin. An iterative regularization method for total variation-based image restoration[J].MMS, 2005,4:460~489
[20] W Yin,S Osher,D Goldfarb, J Darbon. Bregman iterative algorithms for L1-minimization with applications to compressed sensing[J].SIAM J.Imaging Science,2008,1:142~168.
[21] JF Cai, S. Osher, Z Shen. Linearized Bregman iterations for Compressive Sensing and Sparse Denoising[R/OL].UCLA CAM Report, 2008:08~37
[22] Tom Goldstein, Stanley Osher.The Split Bregman Method for L1 Regularization Problems[R].UCLA CAM Report 08-29, 2008.
[23] Stephen Boyd, Lieven Vandenberghe. Covex Optimization [M]. Cambridge:Cambridge University Press,2004.
[24]万保成,分裂Bregman方法及其在图像处理中的应用,吉林大学硕士学术论文,2009年4月
[25] Tom Goldstein, Xavier Bresson, Stanley Osher. Geometric applications of the split bregman method : segmentation and surface reconstruction 2009
[26]李敏,冯象初.基于全变差和小波方法的图像去噪模型[J].西安电子科技大学学报.2006年12月.
[27] L. Zhang, B. Paul, et al,“Threshold analysis in wavelet-based denoising,”Electronics Letters, vol. 37, pp. 1485-1486, Nov. 2001.
[28] TF Chan, J Shen, HM Zhou. Total Variation Wavelet Inpainting [J]. Journal of Mathematical imaging and Vision, Volume 25, Issue 1(July 2006),Pages: 107– 125.
[29] L.I.Rudin, S.Osher and E. Fatermi, Nonlinear Total Based on Noise Removal Algorithms, Physica D60,259-268,1992
[30] Mumford Shah. Comm Pure Appl Math, XLII, 1989:577-685
[31] Tom Goldstein, Xavier Bresson, Stanley Osher.Geometric Application of the Split Bregman Method: Segmentation and Surface Reconstruction[OL]. http://www.math.ucla.edu/~tagoldst/publications.html.
[32] Stanley Osher, Martin Burger, Donald Goldfarb, Jinjun Xu, and Wotao Yin. An iterative regularization method for total variation-based image restoration[J]. MMS, 4:460–489, 2005.
[33] Efroa A,and Leung T K. Texture Synthesis by Non-parametric Sampling. [EB/OL] psu. Edu/efros99 texture. pdf, 1999-09/2003-12
[34] He X J,Huang D S,Zhang Y,et al. A New Feature of Uniformity of Image Texture Directions Coinciding with the Human Eyes Perception[C]. The First International
[35] S.C.Zhu, Y. Wu, and D. Mumford. Filters, random fields and maximum entropy(frame). International Journal of Computer Vision, 27(2), March/April 1998
[36] D.J.Heeger and J.R.Bergen. Pyramid-based texture analysis/synthesis.In SIGGRAPH’95,pages 229-238,1995.
[37] A. Hirani, T.Totsuka. Combining Frequency and spatial domain information for fast interactive image noise removal. Computer Graphics, SIGGRAPH 96:269-276
[38] E. Simoncelli, J.portilla. Texture characterization via joint statistics of wavelet coefficient magnitudes. 5th IEEE Int’l Conf. on Image Processing, Chicago, IL.Oct 4-7,1998
[39] J. Dorsey, A. Edelman, J. Legakisetal. Modeling and rendering of weathered stone. Prof of ACM SIGGRAPH. 1999(4):225-234
[40] P. Dan, C. Borshukov. Seamless texture mapping of subdivision surfaces by model pelting and texture blending. Proc of ACM SIGGRAPH. 2000(3):471-478
[41] F. Nyret, M. P. Cani. Pattern-based texturing revisited. Proc of ACM SIGGRAPH. 1999(3):235-242
[42] S.C.Zhu, Y.N.Wu and D.B.Mumford. Minimax Entropy Principle and Its Applications to Texture Modelin. Neural Computation. 1997(8):1627-1660
[43] L.Y. Wei,M. Levoy. Fast Texture Synthesis using Tree-structured Vector Quantization. Proceedings of SIGGRAPH 2000.2000(1):479-488
[44] Y. Xu, B. Guo, H.Y. Shum. Chaos mosaic: Fast and Memory Efficient Texture Synthesis. Tech. Rep.MSR-TR. 200(23):456-469
[45] D.J. Heeger and J.R. Bergen. Pyramid-based texture analysis/synthesis.In SIGGRAPH’95. 1995(2):229-238
[46] D. Bonet, Jeremy S. Multi-resolution Sampling Procedure for Analysis and Synthesis of Texture Images. SIGGRAPH’97 Conference Proceedings. 1997:361-368
[47] Besag, J. Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society. 1974(3):192-236
[48] L.Liang, C. Liu, Y. Xu, et al. Real-Time Texture Synthesis by Patch-Based Sampling. Mierosoft Researeh . 2001(40):230-425
[49] A. Criminisi,P. Perez, K. Toyama. Object Removal by Exemplar-Based Inpainting. Proc. Of 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recongnition. 2003(9):18-20
[2] Tony F. Chan, Jianhong Shen.Mathematical Models for Local Non-Texture Inpainting [J].SIAM Journal on Applied Mathematics, 2002, 62:1019-1043.
[3] Tony F. Chan, Jianhong Shen.Non-Texture Inpainting by Curvature-Driven Diffusions (CDD) [R].Technical Report CAM 00-35, Image processing Research Group, UCLA, 2000.
[4] Tsai A. Yezzi J A. Willsky A S. Curve evolution implantation of the Mumford function for image segmentation, denoising, interpolation and magnification[J]. IEEE Transactions on Image Processing, 2001, 10(8): 1169-1186
[5] Chan T F, Kang S H, Shen J H, Euler’s Elastica and Curvature Based Inpainting [J]. SIAM Journal of Applied Mathematics, 2002, 63(2):564-592.
[6] Esedoglu S, Shen J H.Digital Inpainting Based on the Mumford-Shah-Euler Image Model [J]. European Journal on Applied Mathematics, 2002, 13(4):353-370.
[7] Harald Grossauer A Combined PDE and texture Synthesis Approach to Inpainting.
[8]张红英,彭启琮《数字图像修复技术综述》,中国图像图形学报,2007年1月,第12卷第1期
[9]庄建飞,《基于偏微分方程(PDE)的图像修补》学位论文南京理工大学2006-06-01
[10] C.Ballester,M.Bertalmiol,V.Caselles,G.Sapiro.andJ.Vergera.Filling-in by joint interpolation of vector fileds and Gray levels.IEEE Tran.Image Processing,10(8):1200-1211,2001
[11]郑孟琦《图像修补模型、算法及其应用研究》,学位论文,南京理工大学,2005
[13] Chan T F. Shen J H. Non-texture inpainting by curvature-driven diffusions(CDD)[J].Journal of Visual Communication and Image Representation,2001,12(4):436~449.
[14] Rudin I., Osher S, Faterni E. Nonlinear total variation based noise removal algorithms[J]. Physica D, 1992. 60(1~4):259~268
[15] T.F.Chan and J.Shen.Mathematical models for local deterministic inpaintings.SIAM J,Appl.Math,62(3):1019-1043,2001
[16] D.Mumford and J.Shah. Optimal approximations by piecewise smooth functions and associated variational problems. Comm.Pure Applied. Math. 79(12),1988
[17] D.Mumford. Elastica and computer vision. In C. L. Bajaj,editor, Algebraic Geometriy and its Applications, pages 491-506.Springer-Verlag, New York,1994
[18] L Bregman . The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex optimization[J]. USSR Computational Mathematical Physics,1967, 7:200~217
[19] Stanley Osher, Martin Burger, Donalb, Jinjun Xu, Wotao Yin. An iterative regularization method for total variation-based image restoration[J].MMS, 2005,4:460~489
[20] W Yin,S Osher,D Goldfarb, J Darbon. Bregman iterative algorithms for L1-minimization with applications to compressed sensing[J].SIAM J.Imaging Science,2008,1:142~168.
[21] JF Cai, S. Osher, Z Shen. Linearized Bregman iterations for Compressive Sensing and Sparse Denoising[R/OL].UCLA CAM Report, 2008:08~37
[22] Tom Goldstein, Stanley Osher.The Split Bregman Method for L1 Regularization Problems[R].UCLA CAM Report 08-29, 2008.
[23] Stephen Boyd, Lieven Vandenberghe. Covex Optimization [M]. Cambridge:Cambridge University Press,2004.
[24]万保成,分裂Bregman方法及其在图像处理中的应用,吉林大学硕士学术论文,2009年4月
[25] Tom Goldstein, Xavier Bresson, Stanley Osher. Geometric applications of the split bregman method : segmentation and surface reconstruction 2009
[26]李敏,冯象初.基于全变差和小波方法的图像去噪模型[J].西安电子科技大学学报.2006年12月.
[27] L. Zhang, B. Paul, et al,“Threshold analysis in wavelet-based denoising,”Electronics Letters, vol. 37, pp. 1485-1486, Nov. 2001.
[28] TF Chan, J Shen, HM Zhou. Total Variation Wavelet Inpainting [J]. Journal of Mathematical imaging and Vision, Volume 25, Issue 1(July 2006),Pages: 107– 125.
[29] L.I.Rudin, S.Osher and E. Fatermi, Nonlinear Total Based on Noise Removal Algorithms, Physica D60,259-268,1992
[30] Mumford Shah. Comm Pure Appl Math, XLII, 1989:577-685
[31] Tom Goldstein, Xavier Bresson, Stanley Osher.Geometric Application of the Split Bregman Method: Segmentation and Surface Reconstruction[OL]. http://www.math.ucla.edu/~tagoldst/publications.html.
[32] Stanley Osher, Martin Burger, Donald Goldfarb, Jinjun Xu, and Wotao Yin. An iterative regularization method for total variation-based image restoration[J]. MMS, 4:460–489, 2005.
[33] Efroa A,and Leung T K. Texture Synthesis by Non-parametric Sampling. [EB/OL] psu. Edu/efros99 texture. pdf, 1999-09/2003-12
[34] He X J,Huang D S,Zhang Y,et al. A New Feature of Uniformity of Image Texture Directions Coinciding with the Human Eyes Perception[C]. The First International
[35] S.C.Zhu, Y. Wu, and D. Mumford. Filters, random fields and maximum entropy(frame). International Journal of Computer Vision, 27(2), March/April 1998
[36] D.J.Heeger and J.R.Bergen. Pyramid-based texture analysis/synthesis.In SIGGRAPH’95,pages 229-238,1995.
[37] A. Hirani, T.Totsuka. Combining Frequency and spatial domain information for fast interactive image noise removal. Computer Graphics, SIGGRAPH 96:269-276
[38] E. Simoncelli, J.portilla. Texture characterization via joint statistics of wavelet coefficient magnitudes. 5th IEEE Int’l Conf. on Image Processing, Chicago, IL.Oct 4-7,1998
[39] J. Dorsey, A. Edelman, J. Legakisetal. Modeling and rendering of weathered stone. Prof of ACM SIGGRAPH. 1999(4):225-234
[40] P. Dan, C. Borshukov. Seamless texture mapping of subdivision surfaces by model pelting and texture blending. Proc of ACM SIGGRAPH. 2000(3):471-478
[41] F. Nyret, M. P. Cani. Pattern-based texturing revisited. Proc of ACM SIGGRAPH. 1999(3):235-242
[42] S.C.Zhu, Y.N.Wu and D.B.Mumford. Minimax Entropy Principle and Its Applications to Texture Modelin. Neural Computation. 1997(8):1627-1660
[43] L.Y. Wei,M. Levoy. Fast Texture Synthesis using Tree-structured Vector Quantization. Proceedings of SIGGRAPH 2000.2000(1):479-488
[44] Y. Xu, B. Guo, H.Y. Shum. Chaos mosaic: Fast and Memory Efficient Texture Synthesis. Tech. Rep.MSR-TR. 200(23):456-469
[45] D.J. Heeger and J.R. Bergen. Pyramid-based texture analysis/synthesis.In SIGGRAPH’95. 1995(2):229-238
[46] D. Bonet, Jeremy S. Multi-resolution Sampling Procedure for Analysis and Synthesis of Texture Images. SIGGRAPH’97 Conference Proceedings. 1997:361-368
[47] Besag, J. Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society. 1974(3):192-236
[48] L.Liang, C. Liu, Y. Xu, et al. Real-Time Texture Synthesis by Patch-Based Sampling. Mierosoft Researeh . 2001(40):230-425
[49] A. Criminisi,P. Perez, K. Toyama. Object Removal by Exemplar-Based Inpainting. Proc. Of 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recongnition. 2003(9):18-20