基于双目视觉的图像三维重建
|
摘要
本文研究了同一场景不同视点的两幅图像,基于双目视觉,通过小波变换提取特征点,进行像点匹配获取基础矩阵,采用对极几何理论,从而实现三维立体重建。
首先我们介绍了计算机的立体视觉理论,对极几何的基本原理,讨论了基础矩阵求解算法。然后介绍了立体视觉匹配技术,三维重建。立体视觉匹配是计算机视觉和非接触测量研究中最基本的关键问题之一,该技术通过匹配像点获取基础矩阵,从而实现三维立体再现,但是同时图像匹配也是最难彻底解决的问题。本文的图像匹配采用的是特征点匹配,采用小波变换提取图像突变特征点的方法,而且该方法与传统的采用角点方法在性能上作了一些比较。在立体匹配方面,提出了一种良好的特征点匹配方法,首先采用视差方法和区域支持方法求取初始匹配,引入候选匹配原则,从而有效的提高了匹配的正确率,得到初始的匹配点对集合。在此基础上利用基础矩阵对匹配点对集合不断优化,一方面不断去除误配点对,另一方面得到了精确的基础矩阵。
最后我们介绍了立体视觉基于图像的分层化三维重建理论,并且给出了实验结果和分析,并对部分匹配点对进行了射影空间的三维重建。
The two images of the same scene with different viewpoints is studied in this massage. Based on computer vision, with wavelet transformation feature points extraction, fundamental matrix is gotten by feature points matching. The image-based 3D is reconstructed by the theory of epipolar geometry.
At first the theory of computer vision is proposed. The principle of epipolar geometry is introduced. And arithmetic about how to solve fundamental matrix is discussed. Then the theory of vision stereo matching and 3-D reconstruction are introduced.Vision stereo matching is one of the fundamental and significant problems in the study of the computer vision and contactless measurements. This technique makes it possible to reproduce a three-dimensional stereo by getting fundamental matrix through matching pixels. But matching is the most difficult problem to be solved completely. In this article, image is matched by feature points. The characteristic of corner points and feature points extracted by wavelet transformation are presented and compared. In this passage a nicer feature points matching method is proposed. At first, initial feature points pairs are being found with parallax method and area supporting. In this way more correct feature point pairs are founded out and initial feature point pairs aggregation is built up. Then the aggregation is optimized continually through fundamental matrix method. On one hand, error point pairs are moved out. On the other hand, accurate fundamental matrix is built.
At last,the hierarchical structure of 3-D reconstruction technology is introduced. Experiments on the real images show that the algorithm in this work is applicable and effective. And a proportion of matching point pairs is reconstructed in projective space.
首先我们介绍了计算机的立体视觉理论,对极几何的基本原理,讨论了基础矩阵求解算法。然后介绍了立体视觉匹配技术,三维重建。立体视觉匹配是计算机视觉和非接触测量研究中最基本的关键问题之一,该技术通过匹配像点获取基础矩阵,从而实现三维立体再现,但是同时图像匹配也是最难彻底解决的问题。本文的图像匹配采用的是特征点匹配,采用小波变换提取图像突变特征点的方法,而且该方法与传统的采用角点方法在性能上作了一些比较。在立体匹配方面,提出了一种良好的特征点匹配方法,首先采用视差方法和区域支持方法求取初始匹配,引入候选匹配原则,从而有效的提高了匹配的正确率,得到初始的匹配点对集合。在此基础上利用基础矩阵对匹配点对集合不断优化,一方面不断去除误配点对,另一方面得到了精确的基础矩阵。
最后我们介绍了立体视觉基于图像的分层化三维重建理论,并且给出了实验结果和分析,并对部分匹配点对进行了射影空间的三维重建。
The two images of the same scene with different viewpoints is studied in this massage. Based on computer vision, with wavelet transformation feature points extraction, fundamental matrix is gotten by feature points matching. The image-based 3D is reconstructed by the theory of epipolar geometry.
At first the theory of computer vision is proposed. The principle of epipolar geometry is introduced. And arithmetic about how to solve fundamental matrix is discussed. Then the theory of vision stereo matching and 3-D reconstruction are introduced.Vision stereo matching is one of the fundamental and significant problems in the study of the computer vision and contactless measurements. This technique makes it possible to reproduce a three-dimensional stereo by getting fundamental matrix through matching pixels. But matching is the most difficult problem to be solved completely. In this article, image is matched by feature points. The characteristic of corner points and feature points extracted by wavelet transformation are presented and compared. In this passage a nicer feature points matching method is proposed. At first, initial feature points pairs are being found with parallax method and area supporting. In this way more correct feature point pairs are founded out and initial feature point pairs aggregation is built up. Then the aggregation is optimized continually through fundamental matrix method. On one hand, error point pairs are moved out. On the other hand, accurate fundamental matrix is built.
At last,the hierarchical structure of 3-D reconstruction technology is introduced. Experiments on the real images show that the algorithm in this work is applicable and effective. And a proportion of matching point pairs is reconstructed in projective space.
引文
[1] Gabriella Olmo, Enrico Magli. All-Integer Hough Transform: Performance Evaluation[J],Image Processing, Internation Conference[C],2001,338-341.
[2] J.Matas, C.Galambost, J.Kittler. Progressive Probabilistic Hough Transform[J],British Machine Vision Conference,2000~2001.
[3]章毓晋.图像理解与计算机视觉[M],清华大学出版社,2000.
[4] Marr D. Vision. A computational investigation into the human representation and processing of visual information, San Francisco: W.H.Freeman and Company, 1982
[5]马颂德,张正友.计算机视觉-计算理论与算法基础[M],科学出版社,1998.
[6] Cochran, Steven D.,Medioni, Gerard, 3-D surface description from binocular stereo, IEEE trans.PAMI, 1992,14(10):981-994
[7] Faugeraus, O., Laveau,s., Robert,L.,etal. 3D reconstruction of urban scenes from sequences of images. Technical Report, INRIA,1995,2572-2580
[8] Barnard,S.,Fischler,M. Computational stereo, ACM Computer Suiveys,1982,14(4):553-572
[9] Zhenyou Zhang. Flexible camera calibration by viewing a plane from unknown orientation. Corfu,Greece:IEEE Int’l Conf. Computer Vison, 1999.666-673
[10] S. J.Maybank, O.Faugeras. A Theory of Self-Calibration of a Moving Camera. International Journal of Computer Vision, vol.8,1992:123-152
[11] Kitchen. A Rosenfeld Gray-Level Corner Detection. Pattern Recognition Letters, 1982,vol.1:95-102
[12] H. P. Moravec. Towards automatic Visual obstacle avoidance. Cambridge, Massachusetts, USA: The 5th Intenational Joint Conference on Artificial Intelligence, 1977,584-591
[13] S. M. Smith, J. M. Brady. SUSAN-A New Approach to Low Level Image Processing. Journal of Computer Vision,1997,23(1):48-78
[14] C. Schmid, R. Mohr, C. Bauckhage. Comparing and evaluating interest points. Proceeding of the 6th International conference on Computer Vision,1998,230-235
[15] C. Tomasi, T.Kanade Detection and Tracking of Point Features. Tech. Rep. CMUCS-91-132, Carnegie Mellon University,1991.
[16] S. Birchfield, C.Tomasi. A pixel dissimilarity measure that is insensitive to image sampling. IEEE Trans.Pattern Anal. Mach. Intel. ,1998,20(4):401-406
[17] D. Lowe. Object Recognition from Local Scale-Invariant Features. ICCV,1999
[18] A. Baumberg. Reliable Feature Matching Across Widely Separated View. CVPR, 2000
[19] Z. Zhang, R. Deriche, O. Taugeras, Q. T. Luong. A Robust Technique for Matching Two Uncalibrated Images Througn the Recovery of the Unknown Epipolar Geometry. Technical Report, INRIA, 1994.
[20]邓文怡,李乃光,董明利,张靖渝,燕必希.数字摄影测量技术在三维测量中的应用.激光,2001,12(7):697-700.
[21]刘江华,陈佳品,程君实.双目视觉平台的研究,机器人技术与应用[J],Vol.1,2002,36-40.
[22] McMillan, L.. An Image-Based Approach to Three-Dimensional Graphics. Ph. D. Thesis. The Department of Computer Science, The Department of Computer Science, The University of North Carolina: Chapel Hill,1997.
[23] Faugeras, O. Three-Dimensional Computer Vision: A Geometric Viewpoint. Cambridge, MA: MIT Press, 1993.
[24] Richard, H.I. In defence of the 8-point algorithm. In: Grimson, E., ed. Proceedings of the 5th International Conference on Computer Vision. Cambridge: IEEE Computer Science Press, 1995. 1064~1070.
[25]马颂德,张正友.计算机视觉—计算理论与算法基础[M],科学出版社,1998.
[26] R.Hartley. In defence of the 8-point algorithm. Proceedings of 5th ICCV. Cambridge, USA, 1995, 1064-1070
[27] Faugeras O D. Three-Dimensional Computer Vision: A Geometric Viewpoint. Cambridge: MIT Press,1993
[28] Faugeras O D. What can be seen in three dimensions with an uncalibrated stereo rig Computer Vision-ECCV’92, LNCS-Series Vol.588. Barlin: Springer-Verlag, 1992, 563-578
[29] Boufama B, Mohr R. Epipole and fundamental matrix estimation using virturl parallax. In: Proceeding of the 5th ICCV. Cambridge, USA: IEEE Computer Society Press, 1995,1030-1036
[30] Torr P H S, Zisserman A, Maybank S J. Robust detection of degenerate configurations for the fundamental matrix. In: Proceedings of the 5th ICCV. Cambridge, USA: IEEE Computer Society Press,1995,1037-1042
[31] Longust-Higgins H C. A computer algorithm for constructing a scene from two projections. Nature, 1998,293: 133-135
[32] Hartley R. In defence of the 8-point algorithm. In: Proceedings of the 5th ICCV. Cambridge, USA: IEEE Computer Society Press, 1995,1064-1070
[33]吴成柯,彦饶平,卢朝阳.对极几何约束下的运动估计和补偿.电子学报,1998,26(10):66-70
[34] M.Fischler,R.Bolles.Random sample consensus:a paradign for model fitting with application to image analysis and automated cartography[J].Communications of the ACM,1981.24(6):381-395.
[35] Q.T.Luong,R.Deriche,O.Faugeras.On determining the fundamental matrix:analysis of different methods and experimental results[R].INRIA,France,Report of research,1993,No:1894.
[36]王伟,博士论文,西安电子科技大学,序列图像间的几何约束及其应用.1998.
[37] Hartley R I. Projective reconstruction and invariants from multiple images[C]. IEEE TransonPAMI, 1994, 16(10):1036—1041
[38] Z G Zeng, H Yan. Region matching and optimal matching pair theorem. Computer Graphics International 2001
[39] T Kanade, M Okutomi. A stereo matching algorithm with an adaptive window. Theory and Experiment, 1994, 16(9): 920-931
[40] D Rosenholm. Multi-point matching using the least-squares technique for an evaluation of three-dimensional models. Photogrammatic Engineering and Remote Sensing, 1987,53(6): 1214-1218
[41] F Schaffalitzky, A Zisserman. Viewpoint invariant texture matching and wide baseline stereo. IEEE. 2001
[42] Raymond van Ee, Clifton M Schor. Unconstrained stereoscopic matching of lines. Vision Research, 2000, 40: 151-162
[43] S J D Prince, R D Eagle. Weighted directional energy model for human stereo correspondence. Vision Research, 2001,13:41-52
[44] P Moallem, K Faez. Search space reduction in the edge based stereo correspondence. VMV 2001
[45] D J Fleet, A D Jepson, M R M Jenkin. Phase-based Disparity Measurement. CVGIP: Image understanding, 1991, 53(2): 198-210
[46] C D Kuglin, D C Hinse. The phase correlation image alignment method. In:Proc of IEEE Int Conf on Cybernetics and Society,1975: 163-165
[47] L D Cai, J Mayhew. A mote on some phase differencing algorithms for disparity estimation. Int Journal of Computer Vision,1997,22(2):111-124
[48] J Zhou, Y Xu, W R Yu. Phase matching with multiresolution wavelet transform. Proc SPIE, 2002,4661(10): 82-91
[49] LI H, MANJUNATH B S, MITRA S K. A contour-based approach to multisensor image registration. IEEETrans, 1995, ImageProcessing, 1995, 4(3): 320-333.
[50] Zhiqiang Zheng, Han Wang, Eam Khwang Teoh. Analysis of gray level corner detection. Pattern Recognition Letters, 1999, 20: 149-162
[51] Liu Wen-Yu, Li hua, Zhu Guang-Xi. A fast algorithm for corner detection using the morphologic skeleton. Pattern Recognition Letter, 2001, 22: 891-900
[52] Nicu Sebe, Michael S. Lew. Comparing salient point detectors. Pattern Recognition Letters, 2003, 24: 89-96.
[53]周婷婷,基于图像的三维测量[D].安徽:安徽大学电子科学与技术学院,2005.
[54] DAI X, KHORRAM S. A feature-based image registration algorithm using improvedchain-code representation combined with invariant moments. IEEETrans, 1999, Geoscience and Remote Sensing, 1999, 37(5):2351-2362.
[55] FLUSSER J, SUK T. A moment-based approach to registration of Images with affine geometric distortion.IEEE Trans. 1996, Geoscience and Remote Sensing, 1996, 32(2): 382-387.
[56] C. Schmid, R. Mohr, C. Bauckhage. Comparing and evaluating interest points. Proceeding of the 6th International Conference on Computer Vision, 1998,230-235
[57]高文,陈熙霖.计算机视觉-算法与系统原理,清华大学出版社,广西科学技术出版社,2000,4:20-78
[2] J.Matas, C.Galambost, J.Kittler. Progressive Probabilistic Hough Transform[J],British Machine Vision Conference,2000~2001.
[3]章毓晋.图像理解与计算机视觉[M],清华大学出版社,2000.
[4] Marr D. Vision. A computational investigation into the human representation and processing of visual information, San Francisco: W.H.Freeman and Company, 1982
[5]马颂德,张正友.计算机视觉-计算理论与算法基础[M],科学出版社,1998.
[6] Cochran, Steven D.,Medioni, Gerard, 3-D surface description from binocular stereo, IEEE trans.PAMI, 1992,14(10):981-994
[7] Faugeraus, O., Laveau,s., Robert,L.,etal. 3D reconstruction of urban scenes from sequences of images. Technical Report, INRIA,1995,2572-2580
[8] Barnard,S.,Fischler,M. Computational stereo, ACM Computer Suiveys,1982,14(4):553-572
[9] Zhenyou Zhang. Flexible camera calibration by viewing a plane from unknown orientation. Corfu,Greece:IEEE Int’l Conf. Computer Vison, 1999.666-673
[10] S. J.Maybank, O.Faugeras. A Theory of Self-Calibration of a Moving Camera. International Journal of Computer Vision, vol.8,1992:123-152
[11] Kitchen. A Rosenfeld Gray-Level Corner Detection. Pattern Recognition Letters, 1982,vol.1:95-102
[12] H. P. Moravec. Towards automatic Visual obstacle avoidance. Cambridge, Massachusetts, USA: The 5th Intenational Joint Conference on Artificial Intelligence, 1977,584-591
[13] S. M. Smith, J. M. Brady. SUSAN-A New Approach to Low Level Image Processing. Journal of Computer Vision,1997,23(1):48-78
[14] C. Schmid, R. Mohr, C. Bauckhage. Comparing and evaluating interest points. Proceeding of the 6th International conference on Computer Vision,1998,230-235
[15] C. Tomasi, T.Kanade Detection and Tracking of Point Features. Tech. Rep. CMUCS-91-132, Carnegie Mellon University,1991.
[16] S. Birchfield, C.Tomasi. A pixel dissimilarity measure that is insensitive to image sampling. IEEE Trans.Pattern Anal. Mach. Intel. ,1998,20(4):401-406
[17] D. Lowe. Object Recognition from Local Scale-Invariant Features. ICCV,1999
[18] A. Baumberg. Reliable Feature Matching Across Widely Separated View. CVPR, 2000
[19] Z. Zhang, R. Deriche, O. Taugeras, Q. T. Luong. A Robust Technique for Matching Two Uncalibrated Images Througn the Recovery of the Unknown Epipolar Geometry. Technical Report, INRIA, 1994.
[20]邓文怡,李乃光,董明利,张靖渝,燕必希.数字摄影测量技术在三维测量中的应用.激光,2001,12(7):697-700.
[21]刘江华,陈佳品,程君实.双目视觉平台的研究,机器人技术与应用[J],Vol.1,2002,36-40.
[22] McMillan, L.. An Image-Based Approach to Three-Dimensional Graphics. Ph. D. Thesis. The Department of Computer Science, The Department of Computer Science, The University of North Carolina: Chapel Hill,1997.
[23] Faugeras, O. Three-Dimensional Computer Vision: A Geometric Viewpoint. Cambridge, MA: MIT Press, 1993.
[24] Richard, H.I. In defence of the 8-point algorithm. In: Grimson, E., ed. Proceedings of the 5th International Conference on Computer Vision. Cambridge: IEEE Computer Science Press, 1995. 1064~1070.
[25]马颂德,张正友.计算机视觉—计算理论与算法基础[M],科学出版社,1998.
[26] R.Hartley. In defence of the 8-point algorithm. Proceedings of 5th ICCV. Cambridge, USA, 1995, 1064-1070
[27] Faugeras O D. Three-Dimensional Computer Vision: A Geometric Viewpoint. Cambridge: MIT Press,1993
[28] Faugeras O D. What can be seen in three dimensions with an uncalibrated stereo rig Computer Vision-ECCV’92, LNCS-Series Vol.588. Barlin: Springer-Verlag, 1992, 563-578
[29] Boufama B, Mohr R. Epipole and fundamental matrix estimation using virturl parallax. In: Proceeding of the 5th ICCV. Cambridge, USA: IEEE Computer Society Press, 1995,1030-1036
[30] Torr P H S, Zisserman A, Maybank S J. Robust detection of degenerate configurations for the fundamental matrix. In: Proceedings of the 5th ICCV. Cambridge, USA: IEEE Computer Society Press,1995,1037-1042
[31] Longust-Higgins H C. A computer algorithm for constructing a scene from two projections. Nature, 1998,293: 133-135
[32] Hartley R. In defence of the 8-point algorithm. In: Proceedings of the 5th ICCV. Cambridge, USA: IEEE Computer Society Press, 1995,1064-1070
[33]吴成柯,彦饶平,卢朝阳.对极几何约束下的运动估计和补偿.电子学报,1998,26(10):66-70
[34] M.Fischler,R.Bolles.Random sample consensus:a paradign for model fitting with application to image analysis and automated cartography[J].Communications of the ACM,1981.24(6):381-395.
[35] Q.T.Luong,R.Deriche,O.Faugeras.On determining the fundamental matrix:analysis of different methods and experimental results[R].INRIA,France,Report of research,1993,No:1894.
[36]王伟,博士论文,西安电子科技大学,序列图像间的几何约束及其应用.1998.
[37] Hartley R I. Projective reconstruction and invariants from multiple images[C]. IEEE TransonPAMI, 1994, 16(10):1036—1041
[38] Z G Zeng, H Yan. Region matching and optimal matching pair theorem. Computer Graphics International 2001
[39] T Kanade, M Okutomi. A stereo matching algorithm with an adaptive window. Theory and Experiment, 1994, 16(9): 920-931
[40] D Rosenholm. Multi-point matching using the least-squares technique for an evaluation of three-dimensional models. Photogrammatic Engineering and Remote Sensing, 1987,53(6): 1214-1218
[41] F Schaffalitzky, A Zisserman. Viewpoint invariant texture matching and wide baseline stereo. IEEE. 2001
[42] Raymond van Ee, Clifton M Schor. Unconstrained stereoscopic matching of lines. Vision Research, 2000, 40: 151-162
[43] S J D Prince, R D Eagle. Weighted directional energy model for human stereo correspondence. Vision Research, 2001,13:41-52
[44] P Moallem, K Faez. Search space reduction in the edge based stereo correspondence. VMV 2001
[45] D J Fleet, A D Jepson, M R M Jenkin. Phase-based Disparity Measurement. CVGIP: Image understanding, 1991, 53(2): 198-210
[46] C D Kuglin, D C Hinse. The phase correlation image alignment method. In:Proc of IEEE Int Conf on Cybernetics and Society,1975: 163-165
[47] L D Cai, J Mayhew. A mote on some phase differencing algorithms for disparity estimation. Int Journal of Computer Vision,1997,22(2):111-124
[48] J Zhou, Y Xu, W R Yu. Phase matching with multiresolution wavelet transform. Proc SPIE, 2002,4661(10): 82-91
[49] LI H, MANJUNATH B S, MITRA S K. A contour-based approach to multisensor image registration. IEEETrans, 1995, ImageProcessing, 1995, 4(3): 320-333.
[50] Zhiqiang Zheng, Han Wang, Eam Khwang Teoh. Analysis of gray level corner detection. Pattern Recognition Letters, 1999, 20: 149-162
[51] Liu Wen-Yu, Li hua, Zhu Guang-Xi. A fast algorithm for corner detection using the morphologic skeleton. Pattern Recognition Letter, 2001, 22: 891-900
[52] Nicu Sebe, Michael S. Lew. Comparing salient point detectors. Pattern Recognition Letters, 2003, 24: 89-96.
[53]周婷婷,基于图像的三维测量[D].安徽:安徽大学电子科学与技术学院,2005.
[54] DAI X, KHORRAM S. A feature-based image registration algorithm using improvedchain-code representation combined with invariant moments. IEEETrans, 1999, Geoscience and Remote Sensing, 1999, 37(5):2351-2362.
[55] FLUSSER J, SUK T. A moment-based approach to registration of Images with affine geometric distortion.IEEE Trans. 1996, Geoscience and Remote Sensing, 1996, 32(2): 382-387.
[56] C. Schmid, R. Mohr, C. Bauckhage. Comparing and evaluating interest points. Proceeding of the 6th International Conference on Computer Vision, 1998,230-235
[57]高文,陈熙霖.计算机视觉-算法与系统原理,清华大学出版社,广西科学技术出版社,2000,4:20-78