基于偏微分方程的图像修复应用
|
摘要
数字图像修复技术是指针对数字图像中遗失或者损坏的部分，利用未被损坏的图像信息，按照一定的规则填补，以使得修复后的图像接近或达到原图的视觉效果。图像修复技术起源于艺术品修复，用于恢复受到损坏的绘画作品。随着近年来数字媒体的普及，数字图像修复技术也获得了多样化的应用，除了用于修复损坏的图像之外，还被应用于目标移除、超分辨率分析、图像压缩、视频错误隐藏等问题中。
目前图像修复大致可以分为两类：基于偏微分方程(PDE)的修复模型和基于纹理合成的修复模型。
基于偏微分方程的修复模型就是将图像修复过程转化为一系列的偏微分方程或能量泛函模型，从而通过数值迭代和智能优化的方法来处理图像。该算法可以使待修复区域周围的有用信息沿着等照度线自动向内扩散修复图像，在保持图像边缘的基础上同时平滑了噪声。本文主要研究了偏微分方程修复模型中的几种模型：BSCB模型，曲率驱动扩散(CDD)模型，整体变分法(TV)修复，弹性(Elastic)修复和Mumford-Shah模型修复，对一些模型开展数值实验，并作了简要的分析。
基于偏微分方程的修复模型对于小块的破损具有良好的修复效果，但是对于大块的纹理图像中的破损修复效果不好，而采用基于纹理合成的修复方法能得到很好的修复效果，本文同时也简单介绍了三种基于纹理合成修复的模型方法。
最后，我们提出了一种基于TV-Stokes方程的修复方法。该方法分两个步骤。第一步，我们尝试把等照度线方向传播到待修复区域；第二步，我们用一个结合了零散度条件的能量最小化模型来建立非线性的Stokes方程。这两个步骤最后都归为解决一些非线性的偏微分方程组。关于该方法的离散化和具体实验情况我们也给出了详细的说明。我们也把该改进的算法跟传统的TV模型算法修复效果作了比较和分析。
Digital image inpainting is a technique of repairing a partially damaged or missing image. It fills the missing part of an image, to make it look natural after repairing by employing information of the undamaged part according to some rules. Digital image inpainting derives from restoration of artworks, and has been applied to repair ancient paintings. With the wide spread of multimedia, this technique becomes an important task in a variety of applications, such as object removal, super resolution, image compression and video error concealment.
At present the image inpainting can be divided into two kinds: the image inpainting based on partial differential equations (PDEs) and the image inpainting based on texture synthesis.
The main ideal of PDE-based image inpainting model is to change the image inpainting process into a series of PDE s or energy function models, so we can deal with the image with numerical iterations and intelligent optimization methods. This algorithm propagates the useful information which is around the inpainting domain in the direction of the isophotes automatically. It is also smoothing the noise while preserving the edges. In this paper, we mainly study several types of PDE-based image inpainting models: BSCB model, Curvature Driven Diffusion (CDD) model, TV inpainting, elastic inpainting and Mumford-Shah inpainting. We also present simulations on some models and make some simple analysis.
PDE-based inpainting model works well to small damaged blocks, but doesn't have good effect for big damaged blocks inpainting. So we must use the inpainting method based on texture synthesis to solve the problem. In this paper, we also introduce three kinds of models in a simple way.
Finally, we propose a method based on TV-Stokes equation to do digital image inpainting. It takes two steps. In the first step, we try to propagate the isophote directions into the inpainting domain. An energy minimization model combined with the zero divergence condition is used to get a nonlinear Stokes equation. Both steps reduce to the solving of some nonlinear partial differential equations. The discretization and the experimentation of the method are illuminated in details. We also present compare and analysis of the effects between the traditional TV model arid our improved method.
目前图像修复大致可以分为两类：基于偏微分方程(PDE)的修复模型和基于纹理合成的修复模型。
基于偏微分方程的修复模型就是将图像修复过程转化为一系列的偏微分方程或能量泛函模型，从而通过数值迭代和智能优化的方法来处理图像。该算法可以使待修复区域周围的有用信息沿着等照度线自动向内扩散修复图像，在保持图像边缘的基础上同时平滑了噪声。本文主要研究了偏微分方程修复模型中的几种模型：BSCB模型，曲率驱动扩散(CDD)模型，整体变分法(TV)修复，弹性(Elastic)修复和Mumford-Shah模型修复，对一些模型开展数值实验，并作了简要的分析。
基于偏微分方程的修复模型对于小块的破损具有良好的修复效果，但是对于大块的纹理图像中的破损修复效果不好，而采用基于纹理合成的修复方法能得到很好的修复效果，本文同时也简单介绍了三种基于纹理合成修复的模型方法。
最后，我们提出了一种基于TV-Stokes方程的修复方法。该方法分两个步骤。第一步，我们尝试把等照度线方向传播到待修复区域；第二步，我们用一个结合了零散度条件的能量最小化模型来建立非线性的Stokes方程。这两个步骤最后都归为解决一些非线性的偏微分方程组。关于该方法的离散化和具体实验情况我们也给出了详细的说明。我们也把该改进的算法跟传统的TV模型算法修复效果作了比较和分析。
Digital image inpainting is a technique of repairing a partially damaged or missing image. It fills the missing part of an image, to make it look natural after repairing by employing information of the undamaged part according to some rules. Digital image inpainting derives from restoration of artworks, and has been applied to repair ancient paintings. With the wide spread of multimedia, this technique becomes an important task in a variety of applications, such as object removal, super resolution, image compression and video error concealment.
At present the image inpainting can be divided into two kinds: the image inpainting based on partial differential equations (PDEs) and the image inpainting based on texture synthesis.
The main ideal of PDE-based image inpainting model is to change the image inpainting process into a series of PDE s or energy function models, so we can deal with the image with numerical iterations and intelligent optimization methods. This algorithm propagates the useful information which is around the inpainting domain in the direction of the isophotes automatically. It is also smoothing the noise while preserving the edges. In this paper, we mainly study several types of PDE-based image inpainting models: BSCB model, Curvature Driven Diffusion (CDD) model, TV inpainting, elastic inpainting and Mumford-Shah inpainting. We also present simulations on some models and make some simple analysis.
PDE-based inpainting model works well to small damaged blocks, but doesn't have good effect for big damaged blocks inpainting. So we must use the inpainting method based on texture synthesis to solve the problem. In this paper, we also introduce three kinds of models in a simple way.
Finally, we propose a method based on TV-Stokes equation to do digital image inpainting. It takes two steps. In the first step, we try to propagate the isophote directions into the inpainting domain. An energy minimization model combined with the zero divergence condition is used to get a nonlinear Stokes equation. Both steps reduce to the solving of some nonlinear partial differential equations. The discretization and the experimentation of the method are illuminated in details. We also present compare and analysis of the effects between the traditional TV model arid our improved method.
引文
[1] 黄煦涛．二维数字信号处理Ⅱ[M]．北京：科学出版社，1985．
[2] Gao Makoto, Matsuyama Takashi. Edge Detection Using Anisotropic Diffusion [J]. IEEE Trans. Pattern Anal Machine Intel, 1979, 9: 394～408.
[3] J. J. Koenderink. The Structure of Image [J]. Biological Cybernetics, 1984, 50: 363～371.
[4] M. Bertalmio, G. Sapiro, C. Ballester, V. Caselles. "Image inpainting" [C]. Proc. of the ACM SIGGRAPH 2000. New Orleans: ACM, 2000: 417～424.
[5] T. than, J. Shen. Non-texture Inpainting By Curvature-driven Diffusions (ODD) [J]. Journal of Visual Communication and Image Representation, 2001, 12(4): 436～449.
[6] 张直，陈刚．基于偏微分方程的图像处理[M]．北京：高等教育出版社，2004．
[7] A. P. Witkin. Scale-Space Filtering [C]. Proceedings of the IJC on AI. New York: ACM Inc, 1983: 1019～1021.
[8] T. Chan, J. Shen. Mathematical Models for Local Non-texture Inpaintings [J]. SIAM. Applied Mathematics, 2001, 62(3): 1019～1043.
[9] A. E. H. Love. A Treatise on the Mathematical Theory of Elasticity 4th ed. [M]. New York: Dover Publications, 1944.
[10] D. Mumford, J. Shah. Optimal Approximations by Piecewise Smooth Functions and Associated Variational Problems [J]. Communications on Pure and. Applied Mathematics, 1989, 42(5): 577～685.
[11] L. Rudin, S, Osher. Total Variation Based Image Restoration with Free Local Constraints[C]. IEEE International Conference on Image Proc, Austin: IEEE Press, 1994: 31～35.
[12] 叶庆凯，郑应平．变分法及其应用第1版[M]．北京：国防工业出版社，1991．
[13] 徐萃薇．计算方法引论第1版[M]．高等教育出版社，1985．
[14] L. Rudin, S .Osher, E. Fatemi. Nonlinear Total Variation Based Noise Removal Algorithms [J]. Physical D, 1992, 60: 259-268.
[15] J. Dorsey, A. Edelman, J. Legakis, et al. Modeling and Rendering of Weathered Stone [J]. Proc of ACM SIGGRAPH, 1999, 8: 225-234.
[16] P. Dan, G. Borshukov. Seamless Texture Mapping of Subdivision Surfaces by Model Pelting and Texture Blending [C]. Proc of ACM SIGGRAPH. New Orleans: ACM Press, 2000:471-478.
[17] F. Nyret, M. P. Cani. Pattern-based Texturing Revisited [C]. Proc of ACM SIGGRAPH. Los Angeles: ACM Press, 1999:235-242.
[18] S. C. Zhu, Y. N. Wu, D. B. Mumford. Minimax Entropy Principle and Its application to Texture Modeling [J]. Neural Computation, 1997, 8:1627-1660.
[19] A. A. Efros, T. K.Leung. Texture Synthesis by Nonparametric Sampling* [C]. PICCV. Washington DC: IEEE Computer Society, 1999:1033-1038.
[20] L. Y. Wei, M. Leroy. Fast Texture Synthesis Using Tree-structured Vector Quantization[C]. Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM Press, 2000:479-488.
[21] L. Liang, C. Liu, Y. Xu, et al. Real-Time Texture Synthesis by Patch-Based Sampling [J]. ACM Transactions on Graphics, 2001,40: 230-425.
[22] D. J. Heeger, J.R. Bergen. Pyramid-based Texture Analysis/Synthesis [C]. Proceedings of SIGGRAPH 95. 1995: 229-238.
[23] C. Ballester, M. Bertalmio, V. Caselles, G. Sapiro, J. Vender. Filling in by Joint Interpolation of Vector Fields and Gray Levels [J]. IEEE Trans. Image Processing, 2001,10(8):1200-1211.
[24] M. Bertalmio, A. L. Bertozzi, G. Sapiro. Navier-Stokes, Fluid Dynamics and Image and Video Inpainting [C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2001, Hawaii, USA, December 2001:355-362.
[25] M. Lysaker, A. Lundervold, X.C. Tai. Noise Removal Using Fourth-Order Partial Differential Equation with Applications to Medical Magnetic Resonance Images in Space and Time [J]. IEEE Transactions on Image Processing, 2003, 12(12): 1579-1590.
[26] M. Lysaker, Stanley Osher, X. C. Tai. Noise Removal Using Smoothed Normals and Surface Fitting [J]. IEEE Trans. Image Processing, 2004, 13(10): 1345～1457.
[27] Simon Masnou. Disocclusion: A Variational Approach Using Level Lines [J]. IEEE Transactions on Image Processing, 2002, 11 (2): 68～76.
[28] Xue-Cheng Tai. Image Inpainting Using a TV-Stokes Equation [EB/OL]. http://math.ecnu.edu.cn/academia/article_of_image_conference/shanghai-ecnu-06-b.pdf, 2007-03-02.
[29] J. Weickert, B. H. Romney, M. A. Viergever. Efficient and Reliable Schemes for Nonlinear Diffusion Filtering [J]. IEEE Yrans on Image Processing, 1998, (7): 398～409.
[30] 王丽亚，李小平，方凯．纹理图像的特征提取和分类[J]．微电子学与计算机，2005，22：96～102．
[31] A. Levin, A. Zomet, Y. Weiss. Learning How to Inpaint from Global Image Statistics [C]. Nice: IEEE, 2003: 305～312.
[32] S. Esedoglu, J. H. Shen. Digital Inpainting Based on the Mumford-Shah-Euler Image Model [J]. European Journal on Applied Mathematics, 2002, 13(4): 353～370.
[33] Y. Xu, B. Guo, H. Y. Shum. Chaos Mosaic: Fast and Memory Efficient Texture Synthesis [R]. Beijing: MSR-TR32, 2000: 456～469.
[34] 谢美华，王正明．用偏微分方程做图像分析与处理[J]．激光与光电子学进展，2006，42：36～39．
[35] 王丽亚，李小平，方凯．纹理图像的特征提取和分类[J]．微电子学与计算机，2005，22：96～102．
[36] 李莉，曹继华．基于Markov因果邻域系统的纹理综合算法研究[J]．天津职业技术师范学院学报，2004，14：29～31．
[37] 徐飞．MATLAB应用图像处理(第1版) [M]．西安：西安电子科技大学出版社，2002．
[38] 张兆礼，赵春晖，梅晓丹．现代图像处理技术及Malab实现(第1版)[M]．北京：人民邮电出版社，2001．
[39] 谢美华，王正明．用偏微分方程做图像分析与处理[J]．激光与光电子学进展，2006，42：36～39．
[2] Gao Makoto, Matsuyama Takashi. Edge Detection Using Anisotropic Diffusion [J]. IEEE Trans. Pattern Anal Machine Intel, 1979, 9: 394～408.
[3] J. J. Koenderink. The Structure of Image [J]. Biological Cybernetics, 1984, 50: 363～371.
[4] M. Bertalmio, G. Sapiro, C. Ballester, V. Caselles. "Image inpainting" [C]. Proc. of the ACM SIGGRAPH 2000. New Orleans: ACM, 2000: 417～424.
[5] T. than, J. Shen. Non-texture Inpainting By Curvature-driven Diffusions (ODD) [J]. Journal of Visual Communication and Image Representation, 2001, 12(4): 436～449.
[6] 张直，陈刚．基于偏微分方程的图像处理[M]．北京：高等教育出版社，2004．
[7] A. P. Witkin. Scale-Space Filtering [C]. Proceedings of the IJC on AI. New York: ACM Inc, 1983: 1019～1021.
[8] T. Chan, J. Shen. Mathematical Models for Local Non-texture Inpaintings [J]. SIAM. Applied Mathematics, 2001, 62(3): 1019～1043.
[9] A. E. H. Love. A Treatise on the Mathematical Theory of Elasticity 4th ed. [M]. New York: Dover Publications, 1944.
[10] D. Mumford, J. Shah. Optimal Approximations by Piecewise Smooth Functions and Associated Variational Problems [J]. Communications on Pure and. Applied Mathematics, 1989, 42(5): 577～685.
[11] L. Rudin, S, Osher. Total Variation Based Image Restoration with Free Local Constraints[C]. IEEE International Conference on Image Proc, Austin: IEEE Press, 1994: 31～35.
[12] 叶庆凯，郑应平．变分法及其应用第1版[M]．北京：国防工业出版社，1991．
[13] 徐萃薇．计算方法引论第1版[M]．高等教育出版社，1985．
[14] L. Rudin, S .Osher, E. Fatemi. Nonlinear Total Variation Based Noise Removal Algorithms [J]. Physical D, 1992, 60: 259-268.
[15] J. Dorsey, A. Edelman, J. Legakis, et al. Modeling and Rendering of Weathered Stone [J]. Proc of ACM SIGGRAPH, 1999, 8: 225-234.
[16] P. Dan, G. Borshukov. Seamless Texture Mapping of Subdivision Surfaces by Model Pelting and Texture Blending [C]. Proc of ACM SIGGRAPH. New Orleans: ACM Press, 2000:471-478.
[17] F. Nyret, M. P. Cani. Pattern-based Texturing Revisited [C]. Proc of ACM SIGGRAPH. Los Angeles: ACM Press, 1999:235-242.
[18] S. C. Zhu, Y. N. Wu, D. B. Mumford. Minimax Entropy Principle and Its application to Texture Modeling [J]. Neural Computation, 1997, 8:1627-1660.
[19] A. A. Efros, T. K.Leung. Texture Synthesis by Nonparametric Sampling* [C]. PICCV. Washington DC: IEEE Computer Society, 1999:1033-1038.
[20] L. Y. Wei, M. Leroy. Fast Texture Synthesis Using Tree-structured Vector Quantization[C]. Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM Press, 2000:479-488.
[21] L. Liang, C. Liu, Y. Xu, et al. Real-Time Texture Synthesis by Patch-Based Sampling [J]. ACM Transactions on Graphics, 2001,40: 230-425.
[22] D. J. Heeger, J.R. Bergen. Pyramid-based Texture Analysis/Synthesis [C]. Proceedings of SIGGRAPH 95. 1995: 229-238.
[23] C. Ballester, M. Bertalmio, V. Caselles, G. Sapiro, J. Vender. Filling in by Joint Interpolation of Vector Fields and Gray Levels [J]. IEEE Trans. Image Processing, 2001,10(8):1200-1211.
[24] M. Bertalmio, A. L. Bertozzi, G. Sapiro. Navier-Stokes, Fluid Dynamics and Image and Video Inpainting [C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2001, Hawaii, USA, December 2001:355-362.
[25] M. Lysaker, A. Lundervold, X.C. Tai. Noise Removal Using Fourth-Order Partial Differential Equation with Applications to Medical Magnetic Resonance Images in Space and Time [J]. IEEE Transactions on Image Processing, 2003, 12(12): 1579-1590.
[26] M. Lysaker, Stanley Osher, X. C. Tai. Noise Removal Using Smoothed Normals and Surface Fitting [J]. IEEE Trans. Image Processing, 2004, 13(10): 1345～1457.
[27] Simon Masnou. Disocclusion: A Variational Approach Using Level Lines [J]. IEEE Transactions on Image Processing, 2002, 11 (2): 68～76.
[28] Xue-Cheng Tai. Image Inpainting Using a TV-Stokes Equation [EB/OL]. http://math.ecnu.edu.cn/academia/article_of_image_conference/shanghai-ecnu-06-b.pdf, 2007-03-02.
[29] J. Weickert, B. H. Romney, M. A. Viergever. Efficient and Reliable Schemes for Nonlinear Diffusion Filtering [J]. IEEE Yrans on Image Processing, 1998, (7): 398～409.
[30] 王丽亚，李小平，方凯．纹理图像的特征提取和分类[J]．微电子学与计算机，2005，22：96～102．
[31] A. Levin, A. Zomet, Y. Weiss. Learning How to Inpaint from Global Image Statistics [C]. Nice: IEEE, 2003: 305～312.
[32] S. Esedoglu, J. H. Shen. Digital Inpainting Based on the Mumford-Shah-Euler Image Model [J]. European Journal on Applied Mathematics, 2002, 13(4): 353～370.
[33] Y. Xu, B. Guo, H. Y. Shum. Chaos Mosaic: Fast and Memory Efficient Texture Synthesis [R]. Beijing: MSR-TR32, 2000: 456～469.
[34] 谢美华，王正明．用偏微分方程做图像分析与处理[J]．激光与光电子学进展，2006，42：36～39．
[35] 王丽亚，李小平，方凯．纹理图像的特征提取和分类[J]．微电子学与计算机，2005，22：96～102．
[36] 李莉，曹继华．基于Markov因果邻域系统的纹理综合算法研究[J]．天津职业技术师范学院学报，2004，14：29～31．
[37] 徐飞．MATLAB应用图像处理(第1版) [M]．西安：西安电子科技大学出版社，2002．
[38] 张兆礼，赵春晖，梅晓丹．现代图像处理技术及Malab实现(第1版)[M]．北京：人民邮电出版社，2001．
[39] 谢美华，王正明．用偏微分方程做图像分析与处理[J]．激光与光电子学进展，2006，42：36～39．