图像修补中的PDE模型
|
摘要
图像修补近年来已成为国际上备受关注的热点问题。这项技术就是利用受损区域周围的图像信息恢复、填充受损区域的数据。目前最为广泛应用的方法就是将各类PDE模型运用于修补中。这类方法经过不断的改进,已经取得了较为满意的结果,但是都不同程度的存在缺陷。本文首先介绍了图像修补及建模理论,给出低层图像修补的要求与原则,讨论了修补模型中重要的几何信息,包括梯度向量,梯度模,拉氏算子和曲率。其后介绍了两类修补模型,第一类是基于变分的修补模型,重点介绍全变分(TV)修补模型,Euler弹性修补模型,简要介绍了Mumford-Shah修补模型和Mumford-Shah-Euler修补模型。第二类是基于PDE的修补模型,介绍了BSCB修补模型和CDD修补模型。BSCB模型修补过程中为了防止交叉的水平线加入了一个扩散方程进行扩散,数值实现较为复杂。另外CDD模型中的曲率项采用纯量曲率并不能充分的对曲率特征进行描述,因此针对以上问题,本文主要做了以下工作:
1.BSCB模型是分为两个过程对图像进行修补的,即传递与扩散。本文选择了一个各向异性的扩散因子,并将其加入到修补的传递模型中,即将两个过程合并得到了一个传递与扩散同时进行的修补模型,最后给出了模型的修补算法及结果。
2.CDD模型是在TV模型中加入了曲率项以改进对大尺度连接性问题的修补效果,一般采用平面曲率的计算公式,本文对法曲率进行了分析,引入主曲率带入到CDD模型中,将曲率项由二维推广至三维,因为改进后的模型更全面的描述了曲率,因此能够得到较原模型更好的修补效果。
3.基于对Euler弹性模型修补机制的分析,本文将BSCB模型与CDD模型耦合得到新的修补模型,这样便在切线和法线方向都给予有效修补,并且得到良好的修补结果。
4.最后,给出了一个以上各PDE模型的统一形式。
The image inpainting is becoming one of the most positive regarded problems in international during recent years. This technology is using the information around the destroyed area to renew or fill in the missing data. At present, the most used tools are partial differential equations, which had taken a great result after a lot of improvement. But still these partial differential equations have some flaws somewhat. At first, this paper presents the image inpainting theory, the theory of establishing inpainting models and the request and principia of low level image inpainting. And then explains several important geometric information in the inpainting model such as the gradient vector and its module, the Laplace operator and the curvature. We are presenting two classes of inpainting models. One of them is the model based on variation, we give emphasis to the total variation model and the Euler's elastica model, then briefly presents the Mumford-Shah model and the Mumford-Shah-Euler model. Another one is the model based on PDE, including the BSCB model and the CDD model. The BSCB did inpaint besides a complex anisotropic diffusion equation so it's hard to do numerical implement, in addition, the CDD did use the amount curvature and this could not describe the curvature in a good way, so we improve these model in this paper. The primary works we do are
1. The BSCB model is inpainting by two separated processes, which are transference and diffusion. This paper chose an anisotropic diffusion item to add to the inpainting model, and then the model can do transferring and diffusing at one time. At last, we explain the numerical implementation and the result.
2. The CDD model adds the curvature of image to TV model to solve the connection problem, and get a better result. Usually we use the plane curvature equation. We extend the problem from a plane to a surface. We compute with the mean curvature of the main curvatures and get the better result.
3. Based on the Euler' elastic model, we get a new model by coupling the BSCB model and the CDD model, which combines the normal and tangent directions inpainting to get a good result.
4. Given a uniform format of all these models presented before.
1.BSCB模型是分为两个过程对图像进行修补的,即传递与扩散。本文选择了一个各向异性的扩散因子,并将其加入到修补的传递模型中,即将两个过程合并得到了一个传递与扩散同时进行的修补模型,最后给出了模型的修补算法及结果。
2.CDD模型是在TV模型中加入了曲率项以改进对大尺度连接性问题的修补效果,一般采用平面曲率的计算公式,本文对法曲率进行了分析,引入主曲率带入到CDD模型中,将曲率项由二维推广至三维,因为改进后的模型更全面的描述了曲率,因此能够得到较原模型更好的修补效果。
3.基于对Euler弹性模型修补机制的分析,本文将BSCB模型与CDD模型耦合得到新的修补模型,这样便在切线和法线方向都给予有效修补,并且得到良好的修补结果。
4.最后,给出了一个以上各PDE模型的统一形式。
The image inpainting is becoming one of the most positive regarded problems in international during recent years. This technology is using the information around the destroyed area to renew or fill in the missing data. At present, the most used tools are partial differential equations, which had taken a great result after a lot of improvement. But still these partial differential equations have some flaws somewhat. At first, this paper presents the image inpainting theory, the theory of establishing inpainting models and the request and principia of low level image inpainting. And then explains several important geometric information in the inpainting model such as the gradient vector and its module, the Laplace operator and the curvature. We are presenting two classes of inpainting models. One of them is the model based on variation, we give emphasis to the total variation model and the Euler's elastica model, then briefly presents the Mumford-Shah model and the Mumford-Shah-Euler model. Another one is the model based on PDE, including the BSCB model and the CDD model. The BSCB did inpaint besides a complex anisotropic diffusion equation so it's hard to do numerical implement, in addition, the CDD did use the amount curvature and this could not describe the curvature in a good way, so we improve these model in this paper. The primary works we do are
1. The BSCB model is inpainting by two separated processes, which are transference and diffusion. This paper chose an anisotropic diffusion item to add to the inpainting model, and then the model can do transferring and diffusing at one time. At last, we explain the numerical implementation and the result.
2. The CDD model adds the curvature of image to TV model to solve the connection problem, and get a better result. Usually we use the plane curvature equation. We extend the problem from a plane to a surface. We compute with the mean curvature of the main curvatures and get the better result.
3. Based on the Euler' elastic model, we get a new model by coupling the BSCB model and the CDD model, which combines the normal and tangent directions inpainting to get a good result.
4. Given a uniform format of all these models presented before.
引文
[1] T. Chen etc., The Past and Future of Image and Multidimensional Signal Processing, IEEE Signal Processing Magazine, pp: 21-58,Mar, 1998.
[2] T. F. Chan, J. Shen, A Good Image Model Restoration--on the contribution of Rudin-Osher-Fatemi's BV image model, Available at www.math.ucla.edu/～imagers , Mar.2002.
[3] M. Nitzberg, D. Mumford, T. Shiota. Filtering, Segmentation, and Depth, Berlin: Springer-Verlag, 1993.
[4] S. Masnou, M. Morel. Level-based disocclusion. Proceedings of 5th IEEE Int'l Conf. on Image Process. Chicago, 1998(3): 259-263.
[5] M. Bertalmio, G. Sapiro, V. Caselles, C. Ballester. Image inpainting. Technical report. ECE-University of Minnesota. 1999. http://www.iua.upf.es/～mbertalmio/bertalmi.pdf. [6]T. Chan, J. Shen. Non-texture inpainting by curvature driven diffusions (CDD).September 2000.J.Visual Comm. Image Rep. 2001, 12(4):436-449.
[7] V. Caselles, J. M. Morel, C. Sbert. An axiomatic approach to image interpolation. IEEE Trans. Image Processing, 1998, 7(3): 376-386.
[8] T. F. Chan, J. Shen. Morphologically invariant PDE inpaintings. UCLA CAM Report 01-15, Department of Mathematices, 2001. http://www.math.ucla.edu/～imagers/htmls/report.html
[9] S. Geman and D. German, Stochasic relaxation, Gibbs distribution and the Bayesian restoration image. IEEE Tran. PAMI 6(6), November, 1984.
[10] C. Ballester, M. Bertalmiol, V. Caselles, G. Sapiro, and J. Vergera. Filling-in by joint interpolation of vector fields and Gray levels. IEEE Tran. Image Processing, 10(8): 1200-1211,2001.
[11] T. F. Chan and J. Shen. Mathematical models for local deterministic inpaintings. SIAM J. Appl. Math, 62 (3): 10 19-1043, 2001.
[12] S. Masnou and J. M. Morel. Level-lines based disocclusion. Proceedings of 5th IEEE Int'l Conf. on Image Process., Chicago, 3: 259-263, 1998.
[13] Osher, S., and Sethian, J.A., Fronts Propagating with Curvature-Dependent Speed: Algorithms Based on Hamilton-Jacobi Formulations, Journal of Computational Physics, 79, pp. 12-49,1988.
[14] Shantanu D. Rane, Guilermo Sapiro, Marcelo Bertalmio. Structure and Texture Filling-In of Missing Image Blocks in Wireless Transmission and Compressio Applications. IEEE Trans. Image Process.2003, 12(3): 296-303.
[15] M. Bertalmio, A. L. Bertozzi, G. Sapiro, Navier-Stokes. Fluid dynamics, and image and video inpainting, in Proc. IEEE Computer Vision and Pattern Recognition(CVPR) Dec. 2001. http://www.iua.upf.es/～mbertalmio/final-cvpr, pdf.
[16] F. Guichard and J. M. Morel, Areview of P. D. E. Models in Image Processing and Image Analysis, Report of CMLA de ENS Cachan, 2109.
[17] L. I. Rudin, S. Osher and E. Fatermi, Nonlinear Total Variation Based on Noise Removal Algorithms, Physica D60,259-268,1992.
[18] T. F. Chan and J. Shen. Mathematical models for local deterministic inpaintings. SIAM J. Appl. Math, 62(3): 1019-1043, 2001.
[19] G Aubert, P Kornprobst. Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations [J]. Applied Mathematical Sciences, Springer-Verlag, 2001,147.
[20] D. C. Knill, W. Richards. Perception as Bayesian Inference. Cambridge Univ. Press, 1996.
[21] T. F. Chan, J. Shen. Variational restoration of non-flat image features: models and algorithms. SIAM J. Appl. Math. 2001, 61(4): 1338-1361.
[22] L. Rudin, S. Osher, E. Fatemi. Nonlinear total variation based noise removal algorithms. Physical, 1992(60): 259-268.
[23] G. Kanizsa. Organization in Vision. New York: Praeger, 1979.
[24] S. C. Zhu, D. Mumford. Prior learning and Gibbs reaction-diffusion. IEEE Trans. on PAM I. 1997,19(11): 1236-1250.
[25] S. C. Zhu, Y. N. Wu, D. Mumford. Minimax entropy principle and its applications to texture modeling. Neural Computation. 1997, 9: 1627-1660.
[26] H. Igehy, L. Pereira. Image replacement through texture synthesis. Proceedings of IEEE Int. Conf Image Processing, 1997.
[27] L.Y. Wei, d M. Levoy. Fast texture synthesis using tree-structured vector quantization. Proceedings of SIGGRAPH 2000. 2000(6): 479-488.
[28] A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, P. J.W. Rayner. Interpolation of missing data in image sequences. IEEE Trans. Image Process. 1995, 11(4): 1509-1519.
[29] D. Mumford. Geometry Driven Diffusion in Computer Vision, chapter "The Bayesian rationale for energy functionals", Kluwer Academic, 1994:141-153.
[30] W. Gibbs. Elementary Principles of Statistical Mechanics. Yale University Press, 1902.
[31]L.I.Rodin,Image,Numerical Analysis of Singularities and Shock Filters,Ph.D Dissertation,Califomia Ins.Of Tech.,1987.
[32]Mumford Shah.Comm Pure Appl Math,XLⅡ,1989:577-685.
[33]A.Marquina and S.Osher,A new time dependent model based on level set motion for nonlinear deblurring and noise removal,in Scale-Space Theories in Computer Vision,Lecture Notes in Comput.Sci. 1682,M.Nielsen,P.Johansen,O.F.Olsen,and J.Weickert,eds.,1999,Springer-Verlag,New York,pp.429-434.
[34]A Tsai,Jr A Yezzi,A S WiIIsky.Curve evolution implementation of the Mumford-Shah functional for image segmentation denoising,interpolation and magnification[J].IEEE Trans Image Process,2001,10(8):1169-1186.
[35]L Ambrosio,V M Tortorelli.Approximation of functional depending on jumps by elliptic functionals via T-convergence[J].Comm.Pure Appl Math.,1990,43:999-1036.
[36]S Esedoglu,J Shen.Digital inpainting based on the Mumford-Shah-Euler image model[J].UCLA CAM Report 2001-24 at:www.math.ucla.edu/-imagers,European J Appl Math,in press,2002.
[37]E De Giorgi.Frontiere orientate dimisura minima.Sem Mat Scuola Norm Sup Pisa,1961.
[38]P.Perona,J.Malik.Scale-space and edge detection using anisotropic diffusion.1990,IEEE-PAMI 12:629-639.
[39]A,E H Love.A Treatise on the Mathematical Theory of Elasticity[M].Dover,New York,4~(th),1927.
[40]Tony F.Chan and Jianhong Shen.Morphologically Invariant PDE Inpaintings.UCLA CAM Report 2001.http://citeseer.ist.psu.edu/chan0lmorphologically.html
[41]郑孟琦.图像修补模型、算法及其应用研究[M]:[硕士学位论文].南京:南京理工大学工大学计算机应用技术专业,2005
[42]杨新.图像偏微分方程的原理与应用.上海交通大学出版社,2003
[43]陈刚.基于偏微分方程的图像处理.高等教育出版社,2004
[44]孙志忠.偏微分方程数值解法.科学出版社,2005.
[45]高世松,韦志辉.基于偏微分方程的图像修补[M]:[硕士学位论文].南京:南京理工大学应用数学专业,2004
[46]王妮娜.图像修补方法的研究[M]:[硕士学位论文].南京:东南大学生物医学工程专业,2005
[2] T. F. Chan, J. Shen, A Good Image Model Restoration--on the contribution of Rudin-Osher-Fatemi's BV image model, Available at www.math.ucla.edu/～imagers , Mar.2002.
[3] M. Nitzberg, D. Mumford, T. Shiota. Filtering, Segmentation, and Depth, Berlin: Springer-Verlag, 1993.
[4] S. Masnou, M. Morel. Level-based disocclusion. Proceedings of 5th IEEE Int'l Conf. on Image Process. Chicago, 1998(3): 259-263.
[5] M. Bertalmio, G. Sapiro, V. Caselles, C. Ballester. Image inpainting. Technical report. ECE-University of Minnesota. 1999. http://www.iua.upf.es/～mbertalmio/bertalmi.pdf. [6]T. Chan, J. Shen. Non-texture inpainting by curvature driven diffusions (CDD).September 2000.J.Visual Comm. Image Rep. 2001, 12(4):436-449.
[7] V. Caselles, J. M. Morel, C. Sbert. An axiomatic approach to image interpolation. IEEE Trans. Image Processing, 1998, 7(3): 376-386.
[8] T. F. Chan, J. Shen. Morphologically invariant PDE inpaintings. UCLA CAM Report 01-15, Department of Mathematices, 2001. http://www.math.ucla.edu/～imagers/htmls/report.html
[9] S. Geman and D. German, Stochasic relaxation, Gibbs distribution and the Bayesian restoration image. IEEE Tran. PAMI 6(6), November, 1984.
[10] C. Ballester, M. Bertalmiol, V. Caselles, G. Sapiro, and J. Vergera. Filling-in by joint interpolation of vector fields and Gray levels. IEEE Tran. Image Processing, 10(8): 1200-1211,2001.
[11] T. F. Chan and J. Shen. Mathematical models for local deterministic inpaintings. SIAM J. Appl. Math, 62 (3): 10 19-1043, 2001.
[12] S. Masnou and J. M. Morel. Level-lines based disocclusion. Proceedings of 5th IEEE Int'l Conf. on Image Process., Chicago, 3: 259-263, 1998.
[13] Osher, S., and Sethian, J.A., Fronts Propagating with Curvature-Dependent Speed: Algorithms Based on Hamilton-Jacobi Formulations, Journal of Computational Physics, 79, pp. 12-49,1988.
[14] Shantanu D. Rane, Guilermo Sapiro, Marcelo Bertalmio. Structure and Texture Filling-In of Missing Image Blocks in Wireless Transmission and Compressio Applications. IEEE Trans. Image Process.2003, 12(3): 296-303.
[15] M. Bertalmio, A. L. Bertozzi, G. Sapiro, Navier-Stokes. Fluid dynamics, and image and video inpainting, in Proc. IEEE Computer Vision and Pattern Recognition(CVPR) Dec. 2001. http://www.iua.upf.es/～mbertalmio/final-cvpr, pdf.
[16] F. Guichard and J. M. Morel, Areview of P. D. E. Models in Image Processing and Image Analysis, Report of CMLA de ENS Cachan, 2109.
[17] L. I. Rudin, S. Osher and E. Fatermi, Nonlinear Total Variation Based on Noise Removal Algorithms, Physica D60,259-268,1992.
[18] T. F. Chan and J. Shen. Mathematical models for local deterministic inpaintings. SIAM J. Appl. Math, 62(3): 1019-1043, 2001.
[19] G Aubert, P Kornprobst. Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations [J]. Applied Mathematical Sciences, Springer-Verlag, 2001,147.
[20] D. C. Knill, W. Richards. Perception as Bayesian Inference. Cambridge Univ. Press, 1996.
[21] T. F. Chan, J. Shen. Variational restoration of non-flat image features: models and algorithms. SIAM J. Appl. Math. 2001, 61(4): 1338-1361.
[22] L. Rudin, S. Osher, E. Fatemi. Nonlinear total variation based noise removal algorithms. Physical, 1992(60): 259-268.
[23] G. Kanizsa. Organization in Vision. New York: Praeger, 1979.
[24] S. C. Zhu, D. Mumford. Prior learning and Gibbs reaction-diffusion. IEEE Trans. on PAM I. 1997,19(11): 1236-1250.
[25] S. C. Zhu, Y. N. Wu, D. Mumford. Minimax entropy principle and its applications to texture modeling. Neural Computation. 1997, 9: 1627-1660.
[26] H. Igehy, L. Pereira. Image replacement through texture synthesis. Proceedings of IEEE Int. Conf Image Processing, 1997.
[27] L.Y. Wei, d M. Levoy. Fast texture synthesis using tree-structured vector quantization. Proceedings of SIGGRAPH 2000. 2000(6): 479-488.
[28] A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, P. J.W. Rayner. Interpolation of missing data in image sequences. IEEE Trans. Image Process. 1995, 11(4): 1509-1519.
[29] D. Mumford. Geometry Driven Diffusion in Computer Vision, chapter "The Bayesian rationale for energy functionals", Kluwer Academic, 1994:141-153.
[30] W. Gibbs. Elementary Principles of Statistical Mechanics. Yale University Press, 1902.
[31]L.I.Rodin,Image,Numerical Analysis of Singularities and Shock Filters,Ph.D Dissertation,Califomia Ins.Of Tech.,1987.
[32]Mumford Shah.Comm Pure Appl Math,XLⅡ,1989:577-685.
[33]A.Marquina and S.Osher,A new time dependent model based on level set motion for nonlinear deblurring and noise removal,in Scale-Space Theories in Computer Vision,Lecture Notes in Comput.Sci. 1682,M.Nielsen,P.Johansen,O.F.Olsen,and J.Weickert,eds.,1999,Springer-Verlag,New York,pp.429-434.
[34]A Tsai,Jr A Yezzi,A S WiIIsky.Curve evolution implementation of the Mumford-Shah functional for image segmentation denoising,interpolation and magnification[J].IEEE Trans Image Process,2001,10(8):1169-1186.
[35]L Ambrosio,V M Tortorelli.Approximation of functional depending on jumps by elliptic functionals via T-convergence[J].Comm.Pure Appl Math.,1990,43:999-1036.
[36]S Esedoglu,J Shen.Digital inpainting based on the Mumford-Shah-Euler image model[J].UCLA CAM Report 2001-24 at:www.math.ucla.edu/-imagers,European J Appl Math,in press,2002.
[37]E De Giorgi.Frontiere orientate dimisura minima.Sem Mat Scuola Norm Sup Pisa,1961.
[38]P.Perona,J.Malik.Scale-space and edge detection using anisotropic diffusion.1990,IEEE-PAMI 12:629-639.
[39]A,E H Love.A Treatise on the Mathematical Theory of Elasticity[M].Dover,New York,4~(th),1927.
[40]Tony F.Chan and Jianhong Shen.Morphologically Invariant PDE Inpaintings.UCLA CAM Report 2001.http://citeseer.ist.psu.edu/chan0lmorphologically.html
[41]郑孟琦.图像修补模型、算法及其应用研究[M]:[硕士学位论文].南京:南京理工大学工大学计算机应用技术专业,2005
[42]杨新.图像偏微分方程的原理与应用.上海交通大学出版社,2003
[43]陈刚.基于偏微分方程的图像处理.高等教育出版社,2004
[44]孙志忠.偏微分方程数值解法.科学出版社,2005.
[45]高世松,韦志辉.基于偏微分方程的图像修补[M]:[硕士学位论文].南京:南京理工大学应用数学专业,2004
[46]王妮娜.图像修补方法的研究[M]:[硕士学位论文].南京:东南大学生物医学工程专业,2005