基于立体视觉的三维重建技术研究
|
摘要
摄像机是一种非常有用的测量设备,它不仅能产生真实的场景图像,同时能提供所拍摄场景的几何特性等信息。三维重建就是利用图像或其它信息和手段定量研究实物或场景的三维空间特征的技术。目前,三位重建是计算机视觉中最活跃的研究课题之一。该问题的研究成果广泛应用于机器人导航、物体识别、建筑学、考古学、医学和军事等各领域。从理论上说,通过摄像机获取的图像一般为二维的灰度图像,它是三维物体的几何特征、光照、物体材料表面性质、摄像机参数和所拍摄图像的环境等许多因素的函数,由灰度反推以上各种参数是逆问题,这些问题往往都是非线性的。如何对受噪声干扰的二维信息恢复出三维信息是计算机视觉领域的一个研究难点。
立体匹配、摄像机标定及三维重建是基于立体视觉三维重建的三个主要部分,本文重点对立体匹配进行了研究。在立体匹配部分,首先提取子象素级角点作为特征点,有效的提高了定位精度,通过立体图像对特征点之间的相关性得到候选匹配对。然而,由于图像噪声、特征点提取算法以及非匹配点的局部相似性和遮挡的影响,相关匹配中必然存在较多的误匹配。针对此问题,本文采用具有强约束关系的极线几何消除初始匹配对中的误匹配,采用鲁棒的最小中值法计算基础矩阵,恢复极线几何,剔除误匹配,获取新的匹配对。经过以上各个阶段的匹配,基本得到一个精度较高的匹配点集,该点集剔除了误匹配和一些不确定的匹配点对,恢复的极线几何相对也很精确,但同时得到的最终匹配点对也很少,所以可以利用求出的基础矩阵从初始匹配集中提取一些新的、符合极线几何约束的匹配点对,得到更多的匹配点对。本文应用VC++及INTEL提供的专门用于计算机视觉开发的OPENCV库,实现了两幅图像的立体匹配。
摄像机标定是立体视觉中非常重要和困难的一步,标定结果的好坏直接影响着三维重建效果。由于实验条件的限制,本文只是简要介绍了传统的标定方法和自标定方法。
在三维重建方面,通过OPENGL实现空间离散点的重建。
A camera is an extraordinarily useful measuring device which produces not only a realistic picture of the scene, but also provides geometric properties information. At present, 3D reconstruction from images is one of the most active areas in computer vision. Many applications require 3D reconstruction such as robot navigation, object recognition and tracking, architecture, archeology, virtual reality, medical diagnosis, military and etc. Theoretically speaking, image captured by the camera generally produces 2D gray image, it is the function combined by many other factors such as geometrical properties of 3D objects, illumination, surface property of objects, camera parameters and images of the environment. These problems are often non-linear which. How to restore the three-dimension information through the two-dimension images affected by the noise is always difficult in computer vision.
Stereo matching, camera calibration and 3D reconstruction are three parts of 3D reconstruction based on stereo vision. Detailed studies had been carried out on these parts. In stereo matching part, firstly subpixel corners were detected as feature points that improve the location accuracy effectively. Through the correlation between the two stereo images, candidate matching sets are obtained. However, due to the image noise, feature extraction algorithm, as well as local similarity of non-stereo matching points, there must be some false matching between the images. Consequently, epipolar geometry was introdued to eliminate the outliers, the robust method called LMedS had been carried out to compute fundamental matrix, and then recovered the epipolar geometry, removed false matching points, obtained the new matching sets. After the above matching steps, high precise matching point sets were obtained which removed some uncertainty and outliers correctly, epipolar geometry is also relatively accurate. At the same time, the final matching points are much rare. So the fundamental matrix was used to obtain some new matching points according to the epipolar geometry, and got more matching points correctly. In this paper, VC++ and OpenCV library that provided by Intel for the devepment of Computer Vision was used, achieved stereo matching between two images.
Camera calibration is very important and difficult step in stereo vision. calibration results will have a direct impact on the effect of 3D Reconstruction. Because of experimental conditions, this paper only introduced the traditional calibration methods and self-calibration methods.
In the visualization of 3D Reconstruction, this paper finally achieved 3D Reconstruction base on OpenGL.
立体匹配、摄像机标定及三维重建是基于立体视觉三维重建的三个主要部分,本文重点对立体匹配进行了研究。在立体匹配部分,首先提取子象素级角点作为特征点,有效的提高了定位精度,通过立体图像对特征点之间的相关性得到候选匹配对。然而,由于图像噪声、特征点提取算法以及非匹配点的局部相似性和遮挡的影响,相关匹配中必然存在较多的误匹配。针对此问题,本文采用具有强约束关系的极线几何消除初始匹配对中的误匹配,采用鲁棒的最小中值法计算基础矩阵,恢复极线几何,剔除误匹配,获取新的匹配对。经过以上各个阶段的匹配,基本得到一个精度较高的匹配点集,该点集剔除了误匹配和一些不确定的匹配点对,恢复的极线几何相对也很精确,但同时得到的最终匹配点对也很少,所以可以利用求出的基础矩阵从初始匹配集中提取一些新的、符合极线几何约束的匹配点对,得到更多的匹配点对。本文应用VC++及INTEL提供的专门用于计算机视觉开发的OPENCV库,实现了两幅图像的立体匹配。
摄像机标定是立体视觉中非常重要和困难的一步,标定结果的好坏直接影响着三维重建效果。由于实验条件的限制,本文只是简要介绍了传统的标定方法和自标定方法。
在三维重建方面,通过OPENGL实现空间离散点的重建。
A camera is an extraordinarily useful measuring device which produces not only a realistic picture of the scene, but also provides geometric properties information. At present, 3D reconstruction from images is one of the most active areas in computer vision. Many applications require 3D reconstruction such as robot navigation, object recognition and tracking, architecture, archeology, virtual reality, medical diagnosis, military and etc. Theoretically speaking, image captured by the camera generally produces 2D gray image, it is the function combined by many other factors such as geometrical properties of 3D objects, illumination, surface property of objects, camera parameters and images of the environment. These problems are often non-linear which. How to restore the three-dimension information through the two-dimension images affected by the noise is always difficult in computer vision.
Stereo matching, camera calibration and 3D reconstruction are three parts of 3D reconstruction based on stereo vision. Detailed studies had been carried out on these parts. In stereo matching part, firstly subpixel corners were detected as feature points that improve the location accuracy effectively. Through the correlation between the two stereo images, candidate matching sets are obtained. However, due to the image noise, feature extraction algorithm, as well as local similarity of non-stereo matching points, there must be some false matching between the images. Consequently, epipolar geometry was introdued to eliminate the outliers, the robust method called LMedS had been carried out to compute fundamental matrix, and then recovered the epipolar geometry, removed false matching points, obtained the new matching sets. After the above matching steps, high precise matching point sets were obtained which removed some uncertainty and outliers correctly, epipolar geometry is also relatively accurate. At the same time, the final matching points are much rare. So the fundamental matrix was used to obtain some new matching points according to the epipolar geometry, and got more matching points correctly. In this paper, VC++ and OpenCV library that provided by Intel for the devepment of Computer Vision was used, achieved stereo matching between two images.
Camera calibration is very important and difficult step in stereo vision. calibration results will have a direct impact on the effect of 3D Reconstruction. Because of experimental conditions, this paper only introduced the traditional calibration methods and self-calibration methods.
In the visualization of 3D Reconstruction, this paper finally achieved 3D Reconstruction base on OpenGL.
引文
[1]马颂德,张正友.计算机视觉-计算理论和算法基础.北京:科学出版社,1998:5-7,52-55
[2]刘侍刚.高精度三维重建技术的研究:博士学位论文.西安电子科技大学.2005:1
[3]王新宇.基于计算机立体视觉的三维重建:硕士学位论文.中南大学.2004:3-5,35-36
[4] Tomasi C,Kanade T.Shape and motion from image streams under orthography: A factorization method.International Journal of Computer Vision, November 1992,9(2):137-154
[5] Beardsley P,Torr P,Zisserman A.3D model acquisition from extended image sequence.In: Cipolla,Roberto,eds. Proceedings of the Fourth European Conference on Computer Vision, Lecture Notes in Computer Science,London:Springs-verlag,1996:683-695
[6] Debevec P,Taylor C ,Malik J.Modeling and rendering architecture from photographs: A hybrid geometry and image based approach.In:Siggraph,eds.Proceedings of the 23th annual conference on Computer Graphics and Interactive Techniques.New York:ACM Press,1996:11-20
[7] Bougnoux S,Robert L.Total Calib: a fast and reliable system for off-line calibration of images sequences.In:Gerard Medioni,Ram Nevatia,eds.Conference on Computer and Pattern Recognition,DEMO Session,SanJuan,Puerto Rico.Washington: IEEE Computer Socierty,1997
[8] Bougnoux S.From Projective to Euclidean Space under any practical situation, a criticism of self-calibration.In:Narendra Ahuja,Uday Desai,eds.Proceedings of the 6th International Conference on Computer Vision.Washington:IEEE Computer Society,1998:790-796
[9] Shum HY,Han M,Szeliski R.Interactive construction of 3D models from panoramicmosaic.In Proceedings of Computer Vision and Pattern Recoginition,1998
[10] Faugeras O,Robert Luc,Laveau S.3D Reconstruction of Urban Scence from Sequences of Images.Computer Vision and Image Understanding,1998,69(3):292~309
[11] Pollefeys M,VanGool L.Visual modeling with a hand-held camera.Accepted for publication in International Journal of Computer Vision
[12] Tsai R.A versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using off-the–shelf TV Camera and Lenses.IEEE Journal of robotics and automation, 1987,3(4):323-344
[13]陈泽志,吴成柯.一种高精度估计的基础矩阵的线性算法*.软件学报.2002,13(4) :840-845
[14] Tsai R Y.An efficient and accurate camera calibration technique for 3D machine vision.Proc.of IEEE Conference of Computer Vision and Pattern Recognition,1987:364-374
[15] Zhang Z.Flextible Camera Calibration by Viewing a Plane from Unknow Oritentations.In Proceedings of the 7th International Conference on Computer Vision,Kerkyra,Greece,1999:666-673
[16] Zhang Z.A Flexible New Technique for Camera Calibration.IEEE Transactions on Pattern Analysis and Machine Intelligence,2000 ,11(22):1330-1334
[17]周东翔,蔡宣平,孙茂印.一种基于特征约束的立体匹配算法.中国图像图形学报.2001,6A(7):653-656
[18]徐光祐.计算机视觉.北京:清华大学出版社,1999
[19] Zhang Z.Image Based Modeling of Objects and Human Faces.SPIE Conference Videometrics and Optical Methods for 3D Shape Measurement.Proceeding of SPIE San Jose,CA,USA,2001(4309):21-26
[20] Pritchett P,Zisserman A.Matching and reconstruction from widely separated views.In 3D Structurefrom Multiple Images of Large-Scale Environments.LNCS 1506.Springer-Verlag.1998
[21] Lhuillier M,Quan L.Robust dense matching using local and global geometric constrains.Proceedings of the IAPR International Conference on Pattern Recognition,Barcelone,Spain,2000 9(2): 968-972
[22] Schmid C,Mohr R,Bauckhage C.Evaluation of interest point detectors[J] .International Journal of Computer Vision,2000,37(2):151-172
[23] Lowe D G.Distinctive image features from scale-invriant keypoints.International Journal of Computer Vision,2004
[24] Asada H,Brady M. The curvature primal sketch.IEEE Transcactions on Pattern Analysis and Machine Intelligence,1986,8:2-14
[25] Deriche R,Faugeras O D.2D-curves matching using high curvatures points :Application to stereovision.In Proc.10th Int’l Conf. Pattern Recog,Atlantic City ,1990,6(1):240-242
[26] Medioni G,Yasumuto Y.Corner detection and curve representation using cubic bspline .In Proc.International Conference on Robotics and Automation,1986:764-769
[27] Horaud R,Veillon F.Finding geometric and relational structures in an image.In Proc. First European Conference On Computer Vision,1990:374-384,
[28] Deriche R,Giraudon G.A computational approach for corner and vertex detection.The International Journal of Computer Vision,1993,10(2):101-124
[29] Beaudet P R.Rotational invariant image opereator.In International Conference Pattern Recognition,1978:333-340
[30] Harris C,Stephens M.A combined corner and edge detector. In Proc. Alvey Conference,1988:189-192
[31] Harris C.Determination of ego-motion from matched points.In proc Alvey Conference,1987
[32] Noble J A. Finding corner.Image and Vision Computing,1998,5(6):121-128
[33] Bouguet J Y.Pyramidal implementation of the lucas Kanade Feature Tracker description of the algorithm
[34] Zhang Z,Deriche R,Faugeras O,Luong Q T.A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence Journal,1995(78):87-119.Also Research Report NO.2273,INRIA Sophia-Antipolis
[35] Torr P,Murray D.The development and comparison of robust methods for estimating the fundamental matrix.International Journal of Computer Vision,1997,24(3):271-300
[36] Longuet-Higgins C.The reconstruction of a scene from two projections:configurations that defeat the 8-point algorithm.In:Proceedings of the 1st Conference on Artificial Intelligence Applications,Denver,1984:395-397
[37] Fichler M A,Bolles R C.Random sample consensus: a paradigm for model fitting with application to image analysis and automated cartography.Graphics and Image Processing,1981,26(4):381-395
[38] Torr PHS,Zisserman A.Mlesac: A new robust estimator with application to estimating image geometry [J] .Computer Vision and Image Understanding,2000,(78):138-156
[39] Rousseeuw R J,Leroy A M.Robust Regression and Outlier Detection.John Wiley Sons,New York,1987
[40] Visual Geometry Group of Oxford,http://www.robots.ox.ac.uk/~vgg/data2.html
[2]刘侍刚.高精度三维重建技术的研究:博士学位论文.西安电子科技大学.2005:1
[3]王新宇.基于计算机立体视觉的三维重建:硕士学位论文.中南大学.2004:3-5,35-36
[4] Tomasi C,Kanade T.Shape and motion from image streams under orthography: A factorization method.International Journal of Computer Vision, November 1992,9(2):137-154
[5] Beardsley P,Torr P,Zisserman A.3D model acquisition from extended image sequence.In: Cipolla,Roberto,eds. Proceedings of the Fourth European Conference on Computer Vision, Lecture Notes in Computer Science,London:Springs-verlag,1996:683-695
[6] Debevec P,Taylor C ,Malik J.Modeling and rendering architecture from photographs: A hybrid geometry and image based approach.In:Siggraph,eds.Proceedings of the 23th annual conference on Computer Graphics and Interactive Techniques.New York:ACM Press,1996:11-20
[7] Bougnoux S,Robert L.Total Calib: a fast and reliable system for off-line calibration of images sequences.In:Gerard Medioni,Ram Nevatia,eds.Conference on Computer and Pattern Recognition,DEMO Session,SanJuan,Puerto Rico.Washington: IEEE Computer Socierty,1997
[8] Bougnoux S.From Projective to Euclidean Space under any practical situation, a criticism of self-calibration.In:Narendra Ahuja,Uday Desai,eds.Proceedings of the 6th International Conference on Computer Vision.Washington:IEEE Computer Society,1998:790-796
[9] Shum HY,Han M,Szeliski R.Interactive construction of 3D models from panoramicmosaic.In Proceedings of Computer Vision and Pattern Recoginition,1998
[10] Faugeras O,Robert Luc,Laveau S.3D Reconstruction of Urban Scence from Sequences of Images.Computer Vision and Image Understanding,1998,69(3):292~309
[11] Pollefeys M,VanGool L.Visual modeling with a hand-held camera.Accepted for publication in International Journal of Computer Vision
[12] Tsai R.A versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using off-the–shelf TV Camera and Lenses.IEEE Journal of robotics and automation, 1987,3(4):323-344
[13]陈泽志,吴成柯.一种高精度估计的基础矩阵的线性算法*.软件学报.2002,13(4) :840-845
[14] Tsai R Y.An efficient and accurate camera calibration technique for 3D machine vision.Proc.of IEEE Conference of Computer Vision and Pattern Recognition,1987:364-374
[15] Zhang Z.Flextible Camera Calibration by Viewing a Plane from Unknow Oritentations.In Proceedings of the 7th International Conference on Computer Vision,Kerkyra,Greece,1999:666-673
[16] Zhang Z.A Flexible New Technique for Camera Calibration.IEEE Transactions on Pattern Analysis and Machine Intelligence,2000 ,11(22):1330-1334
[17]周东翔,蔡宣平,孙茂印.一种基于特征约束的立体匹配算法.中国图像图形学报.2001,6A(7):653-656
[18]徐光祐.计算机视觉.北京:清华大学出版社,1999
[19] Zhang Z.Image Based Modeling of Objects and Human Faces.SPIE Conference Videometrics and Optical Methods for 3D Shape Measurement.Proceeding of SPIE San Jose,CA,USA,2001(4309):21-26
[20] Pritchett P,Zisserman A.Matching and reconstruction from widely separated views.In 3D Structurefrom Multiple Images of Large-Scale Environments.LNCS 1506.Springer-Verlag.1998
[21] Lhuillier M,Quan L.Robust dense matching using local and global geometric constrains.Proceedings of the IAPR International Conference on Pattern Recognition,Barcelone,Spain,2000 9(2): 968-972
[22] Schmid C,Mohr R,Bauckhage C.Evaluation of interest point detectors[J] .International Journal of Computer Vision,2000,37(2):151-172
[23] Lowe D G.Distinctive image features from scale-invriant keypoints.International Journal of Computer Vision,2004
[24] Asada H,Brady M. The curvature primal sketch.IEEE Transcactions on Pattern Analysis and Machine Intelligence,1986,8:2-14
[25] Deriche R,Faugeras O D.2D-curves matching using high curvatures points :Application to stereovision.In Proc.10th Int’l Conf. Pattern Recog,Atlantic City ,1990,6(1):240-242
[26] Medioni G,Yasumuto Y.Corner detection and curve representation using cubic bspline .In Proc.International Conference on Robotics and Automation,1986:764-769
[27] Horaud R,Veillon F.Finding geometric and relational structures in an image.In Proc. First European Conference On Computer Vision,1990:374-384,
[28] Deriche R,Giraudon G.A computational approach for corner and vertex detection.The International Journal of Computer Vision,1993,10(2):101-124
[29] Beaudet P R.Rotational invariant image opereator.In International Conference Pattern Recognition,1978:333-340
[30] Harris C,Stephens M.A combined corner and edge detector. In Proc. Alvey Conference,1988:189-192
[31] Harris C.Determination of ego-motion from matched points.In proc Alvey Conference,1987
[32] Noble J A. Finding corner.Image and Vision Computing,1998,5(6):121-128
[33] Bouguet J Y.Pyramidal implementation of the lucas Kanade Feature Tracker description of the algorithm
[34] Zhang Z,Deriche R,Faugeras O,Luong Q T.A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence Journal,1995(78):87-119.Also Research Report NO.2273,INRIA Sophia-Antipolis
[35] Torr P,Murray D.The development and comparison of robust methods for estimating the fundamental matrix.International Journal of Computer Vision,1997,24(3):271-300
[36] Longuet-Higgins C.The reconstruction of a scene from two projections:configurations that defeat the 8-point algorithm.In:Proceedings of the 1st Conference on Artificial Intelligence Applications,Denver,1984:395-397
[37] Fichler M A,Bolles R C.Random sample consensus: a paradigm for model fitting with application to image analysis and automated cartography.Graphics and Image Processing,1981,26(4):381-395
[38] Torr PHS,Zisserman A.Mlesac: A new robust estimator with application to estimating image geometry [J] .Computer Vision and Image Understanding,2000,(78):138-156
[39] Rousseeuw R J,Leroy A M.Robust Regression and Outlier Detection.John Wiley Sons,New York,1987
[40] Visual Geometry Group of Oxford,http://www.robots.ox.ac.uk/~vgg/data2.html