基于整体变分法的数字图像修复技术研究
|
摘要
数字图像修复是指对数字图像中丢失或破损的部分进行还原修复的过程。通过对破损的图像进行修复更新,使得图像有更好的视觉效果,达到以假乱真的目的。目前,数字图像修复技术在图像压缩、图像编码、文物考古、军事保密等方面有着极为重要的应用。
文章对现有数字图像修复的扩散技术、匹配技术和图像分解技术的发展做了说明,并对其中有代表意义的算法,比如—BSCB模型算法、Criminisi的图像分解法等算法进行原理分析。
本文重点以整体变分法和纹理匹配法作为扩散技术、匹配技术的代表进行研究,并在原有算法的基础上作了一定改进。在整体变分法中加入了相关度系数这一概念,使图像在扩散修复的同时兼顾了各个参考点对目标点不同的影响程度;在纹理匹配法中,对纹理模板块大小和匹配块搜索范围进行了改进说明。
两种改进算法在实验中取得了不错的修复效果。改进后的整体变分法修复后的图像过渡自然,视觉效果有了比较大提高;采用规定了模板大小和匹配块搜索范围的纹理匹配法修复后的图像与周围图像融合的更加真实,而且修复所需时间明显缩短。
本文同时对两种算法各自的适用范围进行了概括。在图像信息丢失较少或者图像边缘信息缺失轻微的情况下,整体变分算法利用边缘扩散技术使得图像更加的平滑;在图像信息丢失较多或者边缘信息缺失严重的情况下,纹理匹配算法利用剩余图像中的相似纹理信息填充修复使得图像更加自然,纹理细节保护的比较好。
Digital image inpainting is a process of restoration in the lost or damaged domain in digital image. In the digital image processing ,to inpaint is to recover the original painting or image ,where the image has been damaged ,to achieve better vision effect so that we even cannot distinguish the original image from the inpainted one .At present, the digital inpainting has played an important role in the image compressing ,image coding, archaeology in the cultural relic and martial secret.
In this paper, We explain the development of the current digital image inpainting techniques such as the diffusion technique ,the technique of matching and the technique of the decomposition .then ,We analyse the theorem of some representative algorithms such as the BSCB model and the decomposition inpainting method created by Criminisi.
We mainly study the total variation method and the texture matching method in this paper as the representatives of the diffusion and matching technology respectively. At the same time ,we make some improvements on the basis of the original algorithm and even add the concept of relativity coefficient into the total variation method. In that case, We can inpaint the image through diffusion while considering the effect of the reference point to the object point. In the texture matching method, the size and the searching area of the template block are improved .
The experiments show that we get good inpainting results. The inpainted image created by the improved total variation algorithm seems natural in the edge and the vision effect of the inpainted image is greatly improved. The inpainted image created by the texture matching method that make use of the set of the size of the texture template and the searching range of the matching block ,fuse better with the surrounding image. Furthermore, the cost of the time is obvious shortened.
In addition, we generalize the applicability of the two algorithms . The experiment results show that when little image information is lost and the edge information is not lost in large extent ,we can apply the improved total variation algorithm and make use of the edge diffusion technology to smooth out the image in order to get better vision effect. Otherwise, when the image or the edge information is lost to a large extent , the texture matching algorithm can fill in the lost information by utilizing the similar texture information in the residual image . The texture matching algorithm make the restored image as natural as its original version and the texture details of the image are also better protected.
文章对现有数字图像修复的扩散技术、匹配技术和图像分解技术的发展做了说明,并对其中有代表意义的算法,比如—BSCB模型算法、Criminisi的图像分解法等算法进行原理分析。
本文重点以整体变分法和纹理匹配法作为扩散技术、匹配技术的代表进行研究,并在原有算法的基础上作了一定改进。在整体变分法中加入了相关度系数这一概念,使图像在扩散修复的同时兼顾了各个参考点对目标点不同的影响程度;在纹理匹配法中,对纹理模板块大小和匹配块搜索范围进行了改进说明。
两种改进算法在实验中取得了不错的修复效果。改进后的整体变分法修复后的图像过渡自然,视觉效果有了比较大提高;采用规定了模板大小和匹配块搜索范围的纹理匹配法修复后的图像与周围图像融合的更加真实,而且修复所需时间明显缩短。
本文同时对两种算法各自的适用范围进行了概括。在图像信息丢失较少或者图像边缘信息缺失轻微的情况下,整体变分算法利用边缘扩散技术使得图像更加的平滑;在图像信息丢失较多或者边缘信息缺失严重的情况下,纹理匹配算法利用剩余图像中的相似纹理信息填充修复使得图像更加自然,纹理细节保护的比较好。
Digital image inpainting is a process of restoration in the lost or damaged domain in digital image. In the digital image processing ,to inpaint is to recover the original painting or image ,where the image has been damaged ,to achieve better vision effect so that we even cannot distinguish the original image from the inpainted one .At present, the digital inpainting has played an important role in the image compressing ,image coding, archaeology in the cultural relic and martial secret.
In this paper, We explain the development of the current digital image inpainting techniques such as the diffusion technique ,the technique of matching and the technique of the decomposition .then ,We analyse the theorem of some representative algorithms such as the BSCB model and the decomposition inpainting method created by Criminisi.
We mainly study the total variation method and the texture matching method in this paper as the representatives of the diffusion and matching technology respectively. At the same time ,we make some improvements on the basis of the original algorithm and even add the concept of relativity coefficient into the total variation method. In that case, We can inpaint the image through diffusion while considering the effect of the reference point to the object point. In the texture matching method, the size and the searching area of the template block are improved .
The experiments show that we get good inpainting results. The inpainted image created by the improved total variation algorithm seems natural in the edge and the vision effect of the inpainted image is greatly improved. The inpainted image created by the texture matching method that make use of the set of the size of the texture template and the searching range of the matching block ,fuse better with the surrounding image. Furthermore, the cost of the time is obvious shortened.
In addition, we generalize the applicability of the two algorithms . The experiment results show that when little image information is lost and the edge information is not lost in large extent ,we can apply the improved total variation algorithm and make use of the edge diffusion technology to smooth out the image in order to get better vision effect. Otherwise, when the image or the edge information is lost to a large extent , the texture matching algorithm can fill in the lost information by utilizing the similar texture information in the residual image . The texture matching algorithm make the restored image as natural as its original version and the texture details of the image are also better protected.
引文
[1]M.Bertalmio,G.Sapiro,V.Caselles and C.Ballester.Image inpainting.In Proc.ACM Conf.Computer.Graphics(SIGGRAPH),New Orleans,LU,Jul 2000,417-424.
[2]T.Chan,J.Shen.Mathematical modles for local Non-texture inpainting [OL].www.math.ucla.edu,2002.
[3]P.M.B van Roosmalen,A.C.Kokaram,J.Biemond.Fast high qualityinterpolation of missing data in image sequences using a controlled pasting scheme[J].IEEE,1999:3105-3108.
[4]Maddalena.Efficient methods for scratch removal in image sequence[J].IEEE,2001:547-552.
[5]T.chan,J shen.Local inpainting models and Tv inpainting[C].SIAM,Apply Math,2001(62):1019-1043.
[6]T.Chan and J.shen.Non-texture inpainting by curvature-driven diffusion [J].Visual Comm.Imager Rep,2001,4(12).
[7]S.Mosnou,M.morel.Level lines based disocclusion[C].Proc IEEE image Processing,Chicago.1998:259-263.
[8]Xue Cheng Tai,Stanley O sher and Randi Holm.Image inpainting using a TV-Stokes equation[J].IEEE Trans,Image Process,13(9),2004.
[9]Sune keller,eg al.A Total Variation Motion Adaptive Deinterlacing Scheme[OL].www.itu.dk,2005.
[10]许巍巍等.一种基于整体变分的图像修复算法[J].中国图像图形学报,2002,7A(4),352-355.
[11]Zhang Quanling,Wang Maoxiang,Wu Lenan.A new algorithm in image re storation[J].Electronic Engineer,1999,25(11):12-14.
[12]Dobson D C,Vogel C R.Convergence of an iterative method for total variation denoising[J].SLAM Journal on Numerical Analysis,1997,34(5):1779-1791.
[13]A.Efros and T.Leung.Texture synthesis by non-parametric sampling[J].In Proc.ICCV,Kerkyra,Greece,Sep 1999,1033-1038.
[14]A.Efros and W.T.Freeman.Image quilting for texture synthesis and transfer.In Proc.ACM Conf.Comp.Graphics(SIGGRAPH),Eugene Fiume,Aug,2001.341-346.
[15]M.Ashikhmin.Synthesizing Natural Texture[J].In Proc.ACM SYmp,On Interative 3D Graphics,2001:217-226.
[16]A.Criminisi,P.Perez,K.Toyama.Region filling and object removal by exemplar-based inpainting[J].In 2004 IEEE Transactions on Image Processing.1200-1212.
[17]M.Bertalmio,L.Vese,G.Saprio,S.Osher.Simultaneous structure and texture image inpainting[J].IEEE Trans.Image Process,2003,10(8).
[18]李厚强,叶中付,詹曙等.一种基于分形和金字塔算法的纹理合成法[J].计算机工程,2001(3):51-53.
[19]Criminisi,Toyama,et,al.Object removal by examplar-based inpainting[J].IEEE Computer vision and pattern Recognition(CVPR),2003:721-728.
[20]Ye Hong.Object Removal Using Exemplar-Based Inpainting.University of Wisconsin-Madison,fall,2004.
[21]Rudin L,Osher S.Nonlinear total variation based noise Removal algorithms[J].Physics D,1992.
[22]L.Rudin and S.osher.Total variation based image restoration with free local constraints[J].IEEE.1 1994,31-35.
[23]祝同江,谈天民.工程数学——变分法[M].北京理工大学出版社,1994.
[24]郑精灵等.整体变分法在图像修复中的应用研究[J].计算机辅助设计与图形学学报,2003,15(10).
[25]邵肖伟,刘政凯,宋璧.一种基于TV模型的自适应图像修复方法[J].电路与系统学报,2004年02期.
[26]郑谋炎,刘小军.总变分图像修复方程的离散化方法[J].电子与信息学报,23(9),2001:861-867.
[27]张红英,彭启琮,吴亚东.数字破损图像的非线性各向异性扩散修复算法[J].计算机辅助设计与图形学学报,2006年10期.79-84.
[28]James H.Money,sung Ha Kang.Total variation minimizing blind deconvolution with shock filter reference[J].Image and Vision Computing,Feb,2008.26卷,第2期.
[29]Shingyu Leung,Stanley Osher.Global Minimization of the Active Contour Model with TV-Inpainting and Two-Phase Denoising[OL].www.math.ucla.edu/～syleung,2007.
[30]Tony F.Chan,Jianhong Shen,Hao-Min Zhou.Total Variation Wavelet Inpainting[J].Journal of Mathematical Imaging and Vision.Jul,2006.2期.
[31]T.F.Chan and S.H.Kang.Error analysis for image inpainting.UCLA CAM Report(04-72),2004.
[32]T.F.Chan and J.Shen.Variatibnal image inpainting[J].Comm.Pure Appl.Math,58(5):579-619,2005.
[33]周密,彭进业,赵健,田艳艳等.改进的整体变分法在图像修复中的应用[J].计算机工程与应用学报,2007年27期.88-91.
[34]Bela Julesz.Visual pattern Discrimination[J].IRE Transcation on information thery,8(2),1962:84-92.
[35]王晨,杜彦,杜建洪.基于块缺失图像修复技术的研究与应用[J].计算机工程,2006年11期.
[36]张红英,彭启琮.数字图像修复技术综述[J].中国图像图形学报,2007,第1期12卷.
[37]ZHANG Hong-ying,PENG Qi-cong.A Survey on Digital Image Inpainting[J].Journal of Image and Graphics,2007,12(1):1-10.
[38]Wang Shu-gen,Zheng Jing-ling.Image inpainting for information lost area based on the texture matching approach[J].Bulletin of Surveying and Mapping,2004,(12):21-23.
[39]Drori,Daniel,Hezy.Fragment based image completion[J].ACM Transactions on Graphics,2003,22(3):303-312.
[40]Zhang Y J,Xiao J G,et al.Region Completion in A Single Image[OL].www.cs.ucf.edu,2005.
[41]Tang F,Ying Y T,Wang J,et al.A novel texture synthesis based algorithm for object removal in photographs[A].In:Proceedings of Ninth Asian Computing Science Conference[C],Chiang Mai,Thailand,2004:248-258.
[2]T.Chan,J.Shen.Mathematical modles for local Non-texture inpainting [OL].www.math.ucla.edu,2002.
[3]P.M.B van Roosmalen,A.C.Kokaram,J.Biemond.Fast high qualityinterpolation of missing data in image sequences using a controlled pasting scheme[J].IEEE,1999:3105-3108.
[4]Maddalena.Efficient methods for scratch removal in image sequence[J].IEEE,2001:547-552.
[5]T.chan,J shen.Local inpainting models and Tv inpainting[C].SIAM,Apply Math,2001(62):1019-1043.
[6]T.Chan and J.shen.Non-texture inpainting by curvature-driven diffusion [J].Visual Comm.Imager Rep,2001,4(12).
[7]S.Mosnou,M.morel.Level lines based disocclusion[C].Proc IEEE image Processing,Chicago.1998:259-263.
[8]Xue Cheng Tai,Stanley O sher and Randi Holm.Image inpainting using a TV-Stokes equation[J].IEEE Trans,Image Process,13(9),2004.
[9]Sune keller,eg al.A Total Variation Motion Adaptive Deinterlacing Scheme[OL].www.itu.dk,2005.
[10]许巍巍等.一种基于整体变分的图像修复算法[J].中国图像图形学报,2002,7A(4),352-355.
[11]Zhang Quanling,Wang Maoxiang,Wu Lenan.A new algorithm in image re storation[J].Electronic Engineer,1999,25(11):12-14.
[12]Dobson D C,Vogel C R.Convergence of an iterative method for total variation denoising[J].SLAM Journal on Numerical Analysis,1997,34(5):1779-1791.
[13]A.Efros and T.Leung.Texture synthesis by non-parametric sampling[J].In Proc.ICCV,Kerkyra,Greece,Sep 1999,1033-1038.
[14]A.Efros and W.T.Freeman.Image quilting for texture synthesis and transfer.In Proc.ACM Conf.Comp.Graphics(SIGGRAPH),Eugene Fiume,Aug,2001.341-346.
[15]M.Ashikhmin.Synthesizing Natural Texture[J].In Proc.ACM SYmp,On Interative 3D Graphics,2001:217-226.
[16]A.Criminisi,P.Perez,K.Toyama.Region filling and object removal by exemplar-based inpainting[J].In 2004 IEEE Transactions on Image Processing.1200-1212.
[17]M.Bertalmio,L.Vese,G.Saprio,S.Osher.Simultaneous structure and texture image inpainting[J].IEEE Trans.Image Process,2003,10(8).
[18]李厚强,叶中付,詹曙等.一种基于分形和金字塔算法的纹理合成法[J].计算机工程,2001(3):51-53.
[19]Criminisi,Toyama,et,al.Object removal by examplar-based inpainting[J].IEEE Computer vision and pattern Recognition(CVPR),2003:721-728.
[20]Ye Hong.Object Removal Using Exemplar-Based Inpainting.University of Wisconsin-Madison,fall,2004.
[21]Rudin L,Osher S.Nonlinear total variation based noise Removal algorithms[J].Physics D,1992.
[22]L.Rudin and S.osher.Total variation based image restoration with free local constraints[J].IEEE.1 1994,31-35.
[23]祝同江,谈天民.工程数学——变分法[M].北京理工大学出版社,1994.
[24]郑精灵等.整体变分法在图像修复中的应用研究[J].计算机辅助设计与图形学学报,2003,15(10).
[25]邵肖伟,刘政凯,宋璧.一种基于TV模型的自适应图像修复方法[J].电路与系统学报,2004年02期.
[26]郑谋炎,刘小军.总变分图像修复方程的离散化方法[J].电子与信息学报,23(9),2001:861-867.
[27]张红英,彭启琮,吴亚东.数字破损图像的非线性各向异性扩散修复算法[J].计算机辅助设计与图形学学报,2006年10期.79-84.
[28]James H.Money,sung Ha Kang.Total variation minimizing blind deconvolution with shock filter reference[J].Image and Vision Computing,Feb,2008.26卷,第2期.
[29]Shingyu Leung,Stanley Osher.Global Minimization of the Active Contour Model with TV-Inpainting and Two-Phase Denoising[OL].www.math.ucla.edu/～syleung,2007.
[30]Tony F.Chan,Jianhong Shen,Hao-Min Zhou.Total Variation Wavelet Inpainting[J].Journal of Mathematical Imaging and Vision.Jul,2006.2期.
[31]T.F.Chan and S.H.Kang.Error analysis for image inpainting.UCLA CAM Report(04-72),2004.
[32]T.F.Chan and J.Shen.Variatibnal image inpainting[J].Comm.Pure Appl.Math,58(5):579-619,2005.
[33]周密,彭进业,赵健,田艳艳等.改进的整体变分法在图像修复中的应用[J].计算机工程与应用学报,2007年27期.88-91.
[34]Bela Julesz.Visual pattern Discrimination[J].IRE Transcation on information thery,8(2),1962:84-92.
[35]王晨,杜彦,杜建洪.基于块缺失图像修复技术的研究与应用[J].计算机工程,2006年11期.
[36]张红英,彭启琮.数字图像修复技术综述[J].中国图像图形学报,2007,第1期12卷.
[37]ZHANG Hong-ying,PENG Qi-cong.A Survey on Digital Image Inpainting[J].Journal of Image and Graphics,2007,12(1):1-10.
[38]Wang Shu-gen,Zheng Jing-ling.Image inpainting for information lost area based on the texture matching approach[J].Bulletin of Surveying and Mapping,2004,(12):21-23.
[39]Drori,Daniel,Hezy.Fragment based image completion[J].ACM Transactions on Graphics,2003,22(3):303-312.
[40]Zhang Y J,Xiao J G,et al.Region Completion in A Single Image[OL].www.cs.ucf.edu,2005.
[41]Tang F,Ying Y T,Wang J,et al.A novel texture synthesis based algorithm for object removal in photographs[A].In:Proceedings of Ninth Asian Computing Science Conference[C],Chiang Mai,Thailand,2004:248-258.