基于偏微分方程的图像修复
|
摘要
图像修复，是利用受损区域周围的图像信息给受损区域填充信息的一门技术，本质上是一种图像插值问题。在旧电影和旧照片的恢复、数字缩放以及电影特效等方面有广泛的应用。
基于偏微分方程的数字图像处理是一个新颖的课题。其在实际应用中的有效性使得越来越多的数学家们关注它。如今，偏微分方程已应用于图像处理和计算机视觉中的许多方面，包括图像分割、运动物体的追踪、物体边缘的探测、图像恢复、图像量化等，大都取得了很好的结果。
本文深入研究了现行的几种图像局部性修复的数学模型和修复算法。研究了基于三次偏微分方程的BSCB模型、基于贝叶斯理论和变分原理的整体变分(Total Variation)修复模型和基于曲率驱动扩散曲(Curvature Driven Diffusions)的CDD模型给出并对比了不同模型的图像修复效果。
本文着重比较了CDD图像修复模型和基于曲率驱动扩散的快速图像修复模型(本文称FCDD)的性能。它们都满足“连接性准则”(connectivity principle)，对具有较大破损区域及细小边缘的图像都具有良好的修复能力；但是前者修复速度较慢。而后者可以克服CDD修复速度慢的缺点。实验结果表明，该模型在保证与CDD模型相近修复质量的情况下可以大幅度地提高修复速度。
Inpainting is to fill in image information on a blank domain based upon the image information available outside, it is a inter polation problem. It has wide applications in restoring scratched old photos, disocclusions in vision analysis, text removal from images, and digital zoomings. Image inpainting is a new area of image restoration. Recently, it is regarded by many researchers in Image Processing area at home and abroad.
PDE-based digital image processing is a new research topic. Because of its efficiency in experiement. more and more mathematicians begin to notice it. Now. PDEs have been widely used in many aspects of image processing and computer visual,including image aeomentation-tracking moving objects, object edges detection, image interpolation and so on; which have been obtained many expected results.
We studied several inpainting methods especially, including the Bertalmio-Sapiro-Caselles-Ballester (BSCB) model and Curvature Driven Diffusions (CDD) model based on PDE, and total variation (TV) model based on the Bayesian and variational principles, also provided different inpainting effect of each model.
This paper puts emphasis on comparing CDD model with fast inpainting model based on the Curvature Driven Diffusions .They all realize the connectivity principle,and have good ability for inpainting the large demain and minute edges, but the former inpainting speed is quite slow.,the latter can overcome CDD model's shortcoming of slow speed. Experiment demonstrates this fast inpainting model can greatly advance the speed in inpainting , moreover, be provided with the same quality off CDD model.
基于偏微分方程的数字图像处理是一个新颖的课题。其在实际应用中的有效性使得越来越多的数学家们关注它。如今，偏微分方程已应用于图像处理和计算机视觉中的许多方面，包括图像分割、运动物体的追踪、物体边缘的探测、图像恢复、图像量化等，大都取得了很好的结果。
本文深入研究了现行的几种图像局部性修复的数学模型和修复算法。研究了基于三次偏微分方程的BSCB模型、基于贝叶斯理论和变分原理的整体变分(Total Variation)修复模型和基于曲率驱动扩散曲(Curvature Driven Diffusions)的CDD模型给出并对比了不同模型的图像修复效果。
本文着重比较了CDD图像修复模型和基于曲率驱动扩散的快速图像修复模型(本文称FCDD)的性能。它们都满足“连接性准则”(connectivity principle)，对具有较大破损区域及细小边缘的图像都具有良好的修复能力；但是前者修复速度较慢。而后者可以克服CDD修复速度慢的缺点。实验结果表明，该模型在保证与CDD模型相近修复质量的情况下可以大幅度地提高修复速度。
Inpainting is to fill in image information on a blank domain based upon the image information available outside, it is a inter polation problem. It has wide applications in restoring scratched old photos, disocclusions in vision analysis, text removal from images, and digital zoomings. Image inpainting is a new area of image restoration. Recently, it is regarded by many researchers in Image Processing area at home and abroad.
PDE-based digital image processing is a new research topic. Because of its efficiency in experiement. more and more mathematicians begin to notice it. Now. PDEs have been widely used in many aspects of image processing and computer visual,including image aeomentation-tracking moving objects, object edges detection, image interpolation and so on; which have been obtained many expected results.
We studied several inpainting methods especially, including the Bertalmio-Sapiro-Caselles-Ballester (BSCB) model and Curvature Driven Diffusions (CDD) model based on PDE, and total variation (TV) model based on the Bayesian and variational principles, also provided different inpainting effect of each model.
This paper puts emphasis on comparing CDD model with fast inpainting model based on the Curvature Driven Diffusions .They all realize the connectivity principle,and have good ability for inpainting the large demain and minute edges, but the former inpainting speed is quite slow.,the latter can overcome CDD model's shortcoming of slow speed. Experiment demonstrates this fast inpainting model can greatly advance the speed in inpainting , moreover, be provided with the same quality off CDD model.
引文
[1] Tony E Chan, Jianhong (Jackie) Shen. Variational Image Inpalnting. UCLA CAM Report 02-63,Department of Mathematices, 2002. http://www.math.ucla.edu/～imagers/htmls/reports.html.
[2] G. Emile-Male. The Restorer's Handbook of Easel Painting. New York: Van Nostrand Reinhold, 1976.
[3] S. Walden. The Ravished Image. New York: St. Martin's Press,1985.
[4] M.Bertalmio, G.Sapiro, V.Caselles, C.Ballester. Image inpainting. Technical report. ECE-University of Minnesota. 1999. http://www.iua.upf. es/～mbertalmio/bertalmi.pdf
[5] V Caselles., J.-M. Morel, C. Sbert. An axiomatic approach to image interpolation. IEEE Trans. Image Processing, 1998,7(3):376-386.
[6] T.Chan, J.Shen. Non-textureinpainting by curvature driven diffusions (CDD). September 2000. J. Visual Comm. Image Rep. 2001,12(4): 436-449.
[7] S. Geman and D. German. Stochastic relaxation, Gibbs distribution and the Bayesian restoration images. IEEE Tran. PAMI-6(6),November, 1984.
[8] Rudin L, Osher S, Fatemi E. Nonlinear total variation based noise removal algorithms. Physica D, 60: 259-268, 1992.
[9] T. F. Chan and J. Shen. Mathematical models for local deterministic inpaintings SIAM J. Appl. Math, 62(3): 1019-1043, 2001.
[10] C. Ballester, M. Bertalmiom Caselles, Cx Sapiro, and J. Vergera. Filling-in by joint interpolation of vector fields and Gray levels. IEEE Tran. Image Processing, 10(8): 1200-1211, 2001.
[11] L. Rudin, S. Osher. Total variation based image restoration with free local constraints. Proc. I st IEEE ICIP1994(1): 31-35.
[12] L. Rudin, S. Osher, E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 1992(60): 259-268.
[13] M. Bertalmio, L. Vese, G Sapiro, and S. Osher. Simultaneous structure and texture image inpainting.
[14] Sung Ha Kang and T. Chan and Stefano Soatto. Landmark based inpainting from multiple views. UCLA Math CAM 02-11,2002.
[15] Tony Chan, Jianhong Shen. Mathematical Models for Local Non-texture Inpaintings. SIAM. Applied Mathematical. 2001,62(3): 1019-1043.
[16] D. Gabon Information Theory in Electron Microscopy. Lab. Invest. 1965 (14) :801-807.
[17] J. J Koenderink. The Structure of Images. Biol, Cyber. 1984(50):363-370.
[18] A. P Witkin. Scale-Space Filtering. Proceedings of the International Joint conference on Artificial Intelligence, ACM Inc. 1983: 1019-1021.
[19] S.Carisson, Sketch. Based coding of gray level images. Signal processing. 1998 (6):5783
[20] Pietro Perona, Jitendra Malik. Scale-Space and Edge Detection Using Anisotropic Diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1990, 12(7): 629-639
[21] S. Osher, J. Sethian. Fronts propagating with curvature dependent speed: based on Hamilton-Jacobi formulations. Journal of Computational algorithms Physics, 1988(79):12-49.
[22] S. Masnou, J.-M. Morel. Level-lines based disocclusion. Proceedings of 5th IEEE Int'lConf. on Image Process. Chicago, 1998(3):259-263.
[23] T. F. Chan, S.-H. Kang, J. Shen. Eider's elastica and curvature based inpaintings. April 2001.SIAM J. Appl. Math. 2002, 63(2): 564-592.
[24] D.Mumford, J.Shan. Optimal approximations by piecewise smooth functions and associated variational problems. Comm.Pure Applied. Math, 1989, XI,II: 577-685.
[25] T. F. Chan and J. Shen. Mathematical models for local deterministic inpaintings ,SIAM J. Appl. Math, 62(3): 1019-1043
[26] S. Geman and D. German. Stochastic relaxation, Gibbs distribution and the Bayesian restoration images. IEEE Tran. PAMI -6(6). November, 1984.
[27] L. Alvarez, P.L. Lions, J.M.Morel. Image selective smoothing and edge detection by nonlinear diffusion. SIAMJ.Numer. Anal. 1992(29}: 845-866.
[28] S. Esedoglu and J. Shen."Digital inpainting based on the Mumford-shah-Euler image model." Technical Report CAM 01-24, Image Processing Research Group, UCLA, 2001.
[29] S. Geman, D. Geman. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Machine Intell. 1984 (6):721-741.
[30] D. C. Knill, W. Richards. Perception as Bayesian Inference. Cambridge Univ. Press, 1996.
[31] D. Mumford. Geometry Driven Diffusion in Computer Vision, chapter "The Bayesian rationale for energy functionals", Kluwer Academic, 1994:141 — 153.
[32] A. Tsai, Jr. A. Yezzi, A. S. Willsky. Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolation and magnification. IEEE Trans. Image Process. 2001, 10(8):1169-1186. [33]
[34] Selim Esedoglu, Jianhong (Jackie) Shen. Digital Inpainting Based on the Mumford-Shah-Euler Txnage Model, September 2001. European J. Appl. Math. 2002(13): 353-370.
[2] G. Emile-Male. The Restorer's Handbook of Easel Painting. New York: Van Nostrand Reinhold, 1976.
[3] S. Walden. The Ravished Image. New York: St. Martin's Press,1985.
[4] M.Bertalmio, G.Sapiro, V.Caselles, C.Ballester. Image inpainting. Technical report. ECE-University of Minnesota. 1999. http://www.iua.upf. es/～mbertalmio/bertalmi.pdf
[5] V Caselles., J.-M. Morel, C. Sbert. An axiomatic approach to image interpolation. IEEE Trans. Image Processing, 1998,7(3):376-386.
[6] T.Chan, J.Shen. Non-textureinpainting by curvature driven diffusions (CDD). September 2000. J. Visual Comm. Image Rep. 2001,12(4): 436-449.
[7] S. Geman and D. German. Stochastic relaxation, Gibbs distribution and the Bayesian restoration images. IEEE Tran. PAMI-6(6),November, 1984.
[8] Rudin L, Osher S, Fatemi E. Nonlinear total variation based noise removal algorithms. Physica D, 60: 259-268, 1992.
[9] T. F. Chan and J. Shen. Mathematical models for local deterministic inpaintings SIAM J. Appl. Math, 62(3): 1019-1043, 2001.
[10] C. Ballester, M. Bertalmiom Caselles, Cx Sapiro, and J. Vergera. Filling-in by joint interpolation of vector fields and Gray levels. IEEE Tran. Image Processing, 10(8): 1200-1211, 2001.
[11] L. Rudin, S. Osher. Total variation based image restoration with free local constraints. Proc. I st IEEE ICIP1994(1): 31-35.
[12] L. Rudin, S. Osher, E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 1992(60): 259-268.
[13] M. Bertalmio, L. Vese, G Sapiro, and S. Osher. Simultaneous structure and texture image inpainting.
[14] Sung Ha Kang and T. Chan and Stefano Soatto. Landmark based inpainting from multiple views. UCLA Math CAM 02-11,2002.
[15] Tony Chan, Jianhong Shen. Mathematical Models for Local Non-texture Inpaintings. SIAM. Applied Mathematical. 2001,62(3): 1019-1043.
[16] D. Gabon Information Theory in Electron Microscopy. Lab. Invest. 1965 (14) :801-807.
[17] J. J Koenderink. The Structure of Images. Biol, Cyber. 1984(50):363-370.
[18] A. P Witkin. Scale-Space Filtering. Proceedings of the International Joint conference on Artificial Intelligence, ACM Inc. 1983: 1019-1021.
[19] S.Carisson, Sketch. Based coding of gray level images. Signal processing. 1998 (6):5783
[20] Pietro Perona, Jitendra Malik. Scale-Space and Edge Detection Using Anisotropic Diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1990, 12(7): 629-639
[21] S. Osher, J. Sethian. Fronts propagating with curvature dependent speed: based on Hamilton-Jacobi formulations. Journal of Computational algorithms Physics, 1988(79):12-49.
[22] S. Masnou, J.-M. Morel. Level-lines based disocclusion. Proceedings of 5th IEEE Int'lConf. on Image Process. Chicago, 1998(3):259-263.
[23] T. F. Chan, S.-H. Kang, J. Shen. Eider's elastica and curvature based inpaintings. April 2001.SIAM J. Appl. Math. 2002, 63(2): 564-592.
[24] D.Mumford, J.Shan. Optimal approximations by piecewise smooth functions and associated variational problems. Comm.Pure Applied. Math, 1989, XI,II: 577-685.
[25] T. F. Chan and J. Shen. Mathematical models for local deterministic inpaintings ,SIAM J. Appl. Math, 62(3): 1019-1043
[26] S. Geman and D. German. Stochastic relaxation, Gibbs distribution and the Bayesian restoration images. IEEE Tran. PAMI -6(6). November, 1984.
[27] L. Alvarez, P.L. Lions, J.M.Morel. Image selective smoothing and edge detection by nonlinear diffusion. SIAMJ.Numer. Anal. 1992(29}: 845-866.
[28] S. Esedoglu and J. Shen."Digital inpainting based on the Mumford-shah-Euler image model." Technical Report CAM 01-24, Image Processing Research Group, UCLA, 2001.
[29] S. Geman, D. Geman. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Machine Intell. 1984 (6):721-741.
[30] D. C. Knill, W. Richards. Perception as Bayesian Inference. Cambridge Univ. Press, 1996.
[31] D. Mumford. Geometry Driven Diffusion in Computer Vision, chapter "The Bayesian rationale for energy functionals", Kluwer Academic, 1994:141 — 153.
[32] A. Tsai, Jr. A. Yezzi, A. S. Willsky. Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolation and magnification. IEEE Trans. Image Process. 2001, 10(8):1169-1186. [33]
[34] Selim Esedoglu, Jianhong (Jackie) Shen. Digital Inpainting Based on the Mumford-Shah-Euler Txnage Model, September 2001. European J. Appl. Math. 2002(13): 353-370.