基于数码相机图像的三维重建技术研究
|
摘要
随着计算机视觉技术和计算机图形学的发展,三维重建技术广泛应用在医学、虚拟现实、计算机动画、三维测量、工业检测以及物理学、化学、地质学等方面。基于数码相机图像的三维重建技术就是从已获取图像中提取所需要的二维信息,并通过相机定标、特征点提取、特征匹配和三维重建算法等相关技术将物体的三维信息恢复出来的过程。
三维重建过程涉及到图像预处理、相机定标、特征点提取、基础矩阵计算等主要问题。在相机定标前,须利用高斯滤波去除图像中的噪声,以利于特征点的正确提取,同时,本研究中还提出了一种平均阈值边缘检测法来检测图像边缘,为相机精确定标奠定了基础;本研究在相机定标的过程中采用了一种新的相机定标方法,即基于线性模型的相机自定标方法,避免了求解过程中的累积误差,有利于高精度求出相机的全部内参数;在基础矩阵求解中,首先利用NCC匹配方法对图像特征点进行粗匹配,然后利用最相关和次相关比例法来消除误匹配点,最后,利用本研究中提出的改进的M-估计法来估计一个精度高和鲁棒性好的基础矩阵;在重建过程中,本研究采用欧氏重建方法求取图像中匹配点的三维空间坐标并通过三角剖分对物体进行建模,使得重建后的物体更具真实感。
本研究的三维图像重建实现以Visual C++和MATLAB为开发工具,采用OpenGL技术显示了三维重建实体,取得了较好的重建效果。实验结果表明,本课题中提出的方法切实可行,实验结果能够反映实体的三维结构,具有较好的真实感。
With the development of computer vision and computer graphics technology, 3D reconstruction technology is widely used in many fields such as medicine, Virtual Reality, computer cartoon, three-dimensional measurement, industry detection, physics, chemistry, and geology. 3D reconstruction technology of image based on digital camera is a process which extracts 2D information from obtained images, and then recovers the object's 3D information on the basis of camera calibration, feature extraction, and feature matching through 3D reconstruction algorithm and some related technology.
3D reconstruction process involves such primitive problems as preprocessing images, calibrating camera, extracting feature and calculating essential matrix. Before camera calibration, it is required that use Gauss wave filters to remove image noise so as to extract minutiae correctly, and also it suggests one kind of average threshold to detect images' edges so that establishes a basis for calibrating camera accurately. This research applies a new camera calibration technology, self-calibration technology based on linear model, which avoids the errors accumulated in the process of calculation, and is well to obtain all the inner parameters of camera accurately.In order to obtain the essential matrix, first of all, NCC match approach is employed to finish the rough match to the image feature points, and then uses nearest and hypo-nearest proportion to remove mistake points, finally it uses improved M-estimate to estimate essential matrix which has a high-accuracy and good robustness. In reconstruction, firstly, Euclidean reconstruction approach is applied to calculate the three-dimensional coordinates of matching points in image, and the entities' model is built through Delaunay triangulation so that make the objects reconstructed more vivid.
This research uses Visual C++, and MATLAB software as development tools to realize 3D reconstruction, and applies OpenGL technology to display 3D object. It gets a good reconstruction effect. The result of experiment shows that the approaches presented in this research is reliable, and the 3D reconstruction model may reveal the structures of entities, in addition, the objects reconstructed have more sense of reality.
三维重建过程涉及到图像预处理、相机定标、特征点提取、基础矩阵计算等主要问题。在相机定标前,须利用高斯滤波去除图像中的噪声,以利于特征点的正确提取,同时,本研究中还提出了一种平均阈值边缘检测法来检测图像边缘,为相机精确定标奠定了基础;本研究在相机定标的过程中采用了一种新的相机定标方法,即基于线性模型的相机自定标方法,避免了求解过程中的累积误差,有利于高精度求出相机的全部内参数;在基础矩阵求解中,首先利用NCC匹配方法对图像特征点进行粗匹配,然后利用最相关和次相关比例法来消除误匹配点,最后,利用本研究中提出的改进的M-估计法来估计一个精度高和鲁棒性好的基础矩阵;在重建过程中,本研究采用欧氏重建方法求取图像中匹配点的三维空间坐标并通过三角剖分对物体进行建模,使得重建后的物体更具真实感。
本研究的三维图像重建实现以Visual C++和MATLAB为开发工具,采用OpenGL技术显示了三维重建实体,取得了较好的重建效果。实验结果表明,本课题中提出的方法切实可行,实验结果能够反映实体的三维结构,具有较好的真实感。
With the development of computer vision and computer graphics technology, 3D reconstruction technology is widely used in many fields such as medicine, Virtual Reality, computer cartoon, three-dimensional measurement, industry detection, physics, chemistry, and geology. 3D reconstruction technology of image based on digital camera is a process which extracts 2D information from obtained images, and then recovers the object's 3D information on the basis of camera calibration, feature extraction, and feature matching through 3D reconstruction algorithm and some related technology.
3D reconstruction process involves such primitive problems as preprocessing images, calibrating camera, extracting feature and calculating essential matrix. Before camera calibration, it is required that use Gauss wave filters to remove image noise so as to extract minutiae correctly, and also it suggests one kind of average threshold to detect images' edges so that establishes a basis for calibrating camera accurately. This research applies a new camera calibration technology, self-calibration technology based on linear model, which avoids the errors accumulated in the process of calculation, and is well to obtain all the inner parameters of camera accurately.In order to obtain the essential matrix, first of all, NCC match approach is employed to finish the rough match to the image feature points, and then uses nearest and hypo-nearest proportion to remove mistake points, finally it uses improved M-estimate to estimate essential matrix which has a high-accuracy and good robustness. In reconstruction, firstly, Euclidean reconstruction approach is applied to calculate the three-dimensional coordinates of matching points in image, and the entities' model is built through Delaunay triangulation so that make the objects reconstructed more vivid.
This research uses Visual C++, and MATLAB software as development tools to realize 3D reconstruction, and applies OpenGL technology to display 3D object. It gets a good reconstruction effect. The result of experiment shows that the approaches presented in this research is reliable, and the 3D reconstruction model may reveal the structures of entities, in addition, the objects reconstructed have more sense of reality.
引文
[1] O.Faugeras.What can be seen in three dimensions with an unealibrated rig? Proc.of 2~(nd) ECCV,Santa Margherita Ligure,Italy Springer-Verlay,May 1992:563-578.
[2] R.Hartly.Invariants of points seen in multiple images.Tech.Rep,GE-CR,Scheneetady,1992.
[3] O.Faugeras, Matusik.Three-Demensional Computer Vision:a geometries viewpoint,The MITPress,Cambridge,Massawhusetts,Lodon,England, 1993.
[4] Laurentini A.The visual hull concept for silhouette based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence,1994,16(2):150~162.
[5] Martin W.N., Aggarwal J.K.Volumetric descriptions of objects from multiple views. IEEE Transactions on Pattern Analysis and Machine Intelligence,1983,5(2): 150-158.
[6] Chien C.H.,Aggarwal J.K. Identification of 3D objects from multiple silhouettes using quadtrees/octrees.Computer Vision,Graphics,and Image Processing, 1986,36: 256-273.
[7] Potmesil M.Generating octree models of 3D objects from their silhouettes in sequence of images.Computer Vision,Graphics,and Image Processing,1987,40(1): 1-29.
[8] Liu Gang.A algorithm to generate visual hulls.Journal of Software,2002,13(9): 1823-1829.
[9] Matusik W.Image-based visual hull.Computer graphics proceedings.Annual Conference Series,ACM SIGGRAPH,New Orleans,Louisiana, 2000: 369-374.
[10] Matusik W.Image-based 3D photography using opacity hulls.In:Computer Graphics Proceedings Annual Conferences Series,ACM SIGGRAPH, San Antonio,Texas,2002:427-437.
[11] Gross M.A spatially immersive display and 3D video portal for telepresence.ACM Transactions on Graphics,2003,22(3):819-827.
[12] Tsai R.A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses.IEEE Journal of Robotics and Automation,1987,3(4): 323-344.
[13]梅向明,刘增贤,林向岩.高等几何.北京:高等教育出版社,1983.
[14]孙即祥,王晓华,钟山等.模式识别中的特征提取与计算机视觉不变量.北京:国防工业出版社,2001.
[15]Tsai R.A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses.IEEE Journal of Robotics and Automation,1987,3(4):323-344.
[16]李弼程,彭天强等.智能图像处理技术.电子工业出版社.2004.7.
[17]Kenneth R.Castlm著,朱志刚等译.数字图像处理.电子工业出版社.2002.2.
[18]D.Marr.Theory of edge detection.Proc.Roy.Soc.Lodon,Vol.B207,1998:187-217.
[19]R.Haralick.Digital step edges from zero-crossing of second directional derivatives.IEEE Trans.Patt.Anal.Machine Intell.,1984,PAMI6(1):58-68.
[20]Tsai R.Y.An Efficient and accurate camera calibration technique for 3D machine vision.IEEE Conference on Computer Vision and Pattern Recognition,1986:36-37.
[21]Pollefeys M.,Koch R.,Vergauwen M.Metric 3D surface reconstruction from uncalibrated image sequences.In Proc.SMILE Workshop,LNCS1506,Springer-Verlag,Berlin,1998:138-153.
[22]Heyden A,Berthilsson R.An iterative factorization method for projective structure and motion from image sequences.Image and Vision Computing,1999,17(13):981-991.
[23]唐泽圣,周嘉玉,李新友编著.计算机图形学基础.清华大学出版社,1997.
[24]Hartley R.Multiple view geometry in computer vision.First published Cambridge University,UK:Cambridge University Press,2000.
[25]陈泽志,吴成柯.一种新的手提相机自定标方法.图像图像学报,2003,8(3):341-346.
[26]A.S.Vasileisks,B.Zhukov,M.Berger,Automated image coregistration based on linear feature recognition.Proeeedings of the Second Conference Fusion of Earth Data,SoPhia AntiPolis,France,1998:59-66.
[27]R.Hartley,B.Zhukov.Multiple view geometry in computer vision.Cambridge University Press,Cambridge,United Kingdom,2000.
[28]张永生,数字摄影测量,北京:解放军出版社,1997.
[29]Longuet-Higgins H.C.A computer algorithm for reconstructing a scene from two projections.Nature,1981,293(10):133-135.
[30] Faugeras, O.What can be seen in three dimensions with an uncalibrated stereo rig In:Sandini,G.,ed.Proceedings of the 2nd European Conference on Computer Vision. Vol.588 of Lecture Notes in Computer Science..Santa Margherita Ligure: Springer-Verlag,1992.
[31] S.M.Smith. Anew approach to low level image processing.Oxford University, 1995.
[32] Moravec H.P. ,Smith. Towards automatic visual obstacle avoidance.In:Proceedings of International Joint Conference on Artificial Intelligence,Cambridge: MIT Press, 1977.
[33] Haddow E R. Anew stereo algorithm. Proc 4th SCIA,1985:175-182.
[34] Shapiro L,Haddow E R.A fast structural matching algorithm with applications in stereo vision.Proc 4~(th) SCIA,1985.
[35] M.Rechsteiner,G..Troester.Dynamic workspace monitoring.International Archives of photo grammetry and emote sensing,Munich,Germany, 1994,30: 689-696.
[36] P.Aschwanden.Experimental results from a comparative study on correlation-type registration algorithms. In:W.Forstner ,Robust Computer Vision,1992:268-289.
[37] O.Faugeras,B.Hotz.Real time correlation-based stereo:Algorithm implementations and applications,Tech. Rep.RR-2013,INRIA, 1993.
[38] S.Carlsson .Multiple image invariance using the double algebra.Computer Vision volume 825 of Lecture Notes in Computer Seience,.Springer-Verlag,1994: 145-164.
[39] P.Beardsley.A sequential updating of projeetive and affine structure from Motion.International Journal of Computer Vision(23), No.3 Jun-Jul 1997: 235-259.
[40] R.Mohr,B.Boufama.Accurate projective reconstruction.Applications of invariance in Computer Vision, Springer-Verlag,1994: 257-276.
[41] C.A.Rothwell ,O.Faugeras. A comparision of projective reconstruction methods for pairs of views. 5th International conference on Computer Vision,Cambrldge,MA,June 1995.
[42] Jonathan Richard Shewchuk.Triangle:Engineering a 2D quality mesh generaror and delaunay triangulator, ACM,May 1996:124-133.
[2] R.Hartly.Invariants of points seen in multiple images.Tech.Rep,GE-CR,Scheneetady,1992.
[3] O.Faugeras, Matusik.Three-Demensional Computer Vision:a geometries viewpoint,The MITPress,Cambridge,Massawhusetts,Lodon,England, 1993.
[4] Laurentini A.The visual hull concept for silhouette based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence,1994,16(2):150~162.
[5] Martin W.N., Aggarwal J.K.Volumetric descriptions of objects from multiple views. IEEE Transactions on Pattern Analysis and Machine Intelligence,1983,5(2): 150-158.
[6] Chien C.H.,Aggarwal J.K. Identification of 3D objects from multiple silhouettes using quadtrees/octrees.Computer Vision,Graphics,and Image Processing, 1986,36: 256-273.
[7] Potmesil M.Generating octree models of 3D objects from their silhouettes in sequence of images.Computer Vision,Graphics,and Image Processing,1987,40(1): 1-29.
[8] Liu Gang.A algorithm to generate visual hulls.Journal of Software,2002,13(9): 1823-1829.
[9] Matusik W.Image-based visual hull.Computer graphics proceedings.Annual Conference Series,ACM SIGGRAPH,New Orleans,Louisiana, 2000: 369-374.
[10] Matusik W.Image-based 3D photography using opacity hulls.In:Computer Graphics Proceedings Annual Conferences Series,ACM SIGGRAPH, San Antonio,Texas,2002:427-437.
[11] Gross M.A spatially immersive display and 3D video portal for telepresence.ACM Transactions on Graphics,2003,22(3):819-827.
[12] Tsai R.A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses.IEEE Journal of Robotics and Automation,1987,3(4): 323-344.
[13]梅向明,刘增贤,林向岩.高等几何.北京:高等教育出版社,1983.
[14]孙即祥,王晓华,钟山等.模式识别中的特征提取与计算机视觉不变量.北京:国防工业出版社,2001.
[15]Tsai R.A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses.IEEE Journal of Robotics and Automation,1987,3(4):323-344.
[16]李弼程,彭天强等.智能图像处理技术.电子工业出版社.2004.7.
[17]Kenneth R.Castlm著,朱志刚等译.数字图像处理.电子工业出版社.2002.2.
[18]D.Marr.Theory of edge detection.Proc.Roy.Soc.Lodon,Vol.B207,1998:187-217.
[19]R.Haralick.Digital step edges from zero-crossing of second directional derivatives.IEEE Trans.Patt.Anal.Machine Intell.,1984,PAMI6(1):58-68.
[20]Tsai R.Y.An Efficient and accurate camera calibration technique for 3D machine vision.IEEE Conference on Computer Vision and Pattern Recognition,1986:36-37.
[21]Pollefeys M.,Koch R.,Vergauwen M.Metric 3D surface reconstruction from uncalibrated image sequences.In Proc.SMILE Workshop,LNCS1506,Springer-Verlag,Berlin,1998:138-153.
[22]Heyden A,Berthilsson R.An iterative factorization method for projective structure and motion from image sequences.Image and Vision Computing,1999,17(13):981-991.
[23]唐泽圣,周嘉玉,李新友编著.计算机图形学基础.清华大学出版社,1997.
[24]Hartley R.Multiple view geometry in computer vision.First published Cambridge University,UK:Cambridge University Press,2000.
[25]陈泽志,吴成柯.一种新的手提相机自定标方法.图像图像学报,2003,8(3):341-346.
[26]A.S.Vasileisks,B.Zhukov,M.Berger,Automated image coregistration based on linear feature recognition.Proeeedings of the Second Conference Fusion of Earth Data,SoPhia AntiPolis,France,1998:59-66.
[27]R.Hartley,B.Zhukov.Multiple view geometry in computer vision.Cambridge University Press,Cambridge,United Kingdom,2000.
[28]张永生,数字摄影测量,北京:解放军出版社,1997.
[29]Longuet-Higgins H.C.A computer algorithm for reconstructing a scene from two projections.Nature,1981,293(10):133-135.
[30] Faugeras, O.What can be seen in three dimensions with an uncalibrated stereo rig In:Sandini,G.,ed.Proceedings of the 2nd European Conference on Computer Vision. Vol.588 of Lecture Notes in Computer Science..Santa Margherita Ligure: Springer-Verlag,1992.
[31] S.M.Smith. Anew approach to low level image processing.Oxford University, 1995.
[32] Moravec H.P. ,Smith. Towards automatic visual obstacle avoidance.In:Proceedings of International Joint Conference on Artificial Intelligence,Cambridge: MIT Press, 1977.
[33] Haddow E R. Anew stereo algorithm. Proc 4th SCIA,1985:175-182.
[34] Shapiro L,Haddow E R.A fast structural matching algorithm with applications in stereo vision.Proc 4~(th) SCIA,1985.
[35] M.Rechsteiner,G..Troester.Dynamic workspace monitoring.International Archives of photo grammetry and emote sensing,Munich,Germany, 1994,30: 689-696.
[36] P.Aschwanden.Experimental results from a comparative study on correlation-type registration algorithms. In:W.Forstner ,Robust Computer Vision,1992:268-289.
[37] O.Faugeras,B.Hotz.Real time correlation-based stereo:Algorithm implementations and applications,Tech. Rep.RR-2013,INRIA, 1993.
[38] S.Carlsson .Multiple image invariance using the double algebra.Computer Vision volume 825 of Lecture Notes in Computer Seience,.Springer-Verlag,1994: 145-164.
[39] P.Beardsley.A sequential updating of projeetive and affine structure from Motion.International Journal of Computer Vision(23), No.3 Jun-Jul 1997: 235-259.
[40] R.Mohr,B.Boufama.Accurate projective reconstruction.Applications of invariance in Computer Vision, Springer-Verlag,1994: 257-276.
[41] C.A.Rothwell ,O.Faugeras. A comparision of projective reconstruction methods for pairs of views. 5th International conference on Computer Vision,Cambrldge,MA,June 1995.
[42] Jonathan Richard Shewchuk.Triangle:Engineering a 2D quality mesh generaror and delaunay triangulator, ACM,May 1996:124-133.