基于三焦点张量的多视图目标三维重建
|
摘要
多视图重建是指从目标物体的多幅不同视角图像中,提取目标物体的三维信息,从而达到重建该物体的目的。目前,多视图重建技术已广泛应用于机器人制造、表面检测、医学影像、手术导航、图像配准等诸多领域。多视图重建技术基于多视角图像,能从至少3张图像中提取有用信息从而实现三维重建。多视图重建过程主要分为射影重构过程和度量校正过程。
目前,射影重构过程中计算三焦点张量多采用系数矩阵奇异值分解方法,该方法容易引入数据冗余,导致解得的三焦点张量几何无效,对后期重建有一定影响。度量校正过程中,传统方法需要摄像机按照特定轨迹移动或者需要图像中具有消失点或平行线等信息,具有一定的应用局限性。针对重构过程中存在的这些问题,本文主要工作如下:
结合对偶化原则,采用基于最小配置实现三焦点张量的计算,从而可避免奇异值分解方法带来的数据冗余,得到几何有效的射影重构。同时结合自适应鲁棒估计和引导匹配算法,可有效去除误匹配干扰,获得较好的特征匹配点和精确的三焦点张量。
采用分层重构的方法,通过解Cheirality不等式计算射影重构中的无穷远平面,从而获得无穷单应,进而计算摄像机的内参数矩阵。在摄像机自标定的基础上将射影重构转换到仿射重构,最终实现度量校正。
与传统的多视图重建算法相比,本文提出的算法能够获得精确的最小配置解,重构时不需要借助消失点、消失线等优点,具有更广泛的应用前景。
Multiple-view reconstruction is extracting 3D information from multiple views of the object, and intent to reconstruction the object. Recent years, academic world has attached attention to Multiple-view reconstruction. This technology is widely used in the field of robot manufacturing, surface inspection, medical imaging, surgical navigation, image registration, and so on. Multiple-view reconstruction is based on multiple views, and can extract useful information from at least three images. The process can be divided into projective reconstruction and metric correction.
Currently, the projective reconstruction commonly uses singular value decomposition of coefficient matrix to compute the trifocal tensor. This method easily leads to geometric invalid solution as it takes data redundancy in. In the process of metric correction, traditional methods are limited in practical application as they need vanishing points, vanishing lines or the special movement of camera. In this thesis, our study and discuss focused on the above two problems.
We computed the trifocal tensor based on minimum case with duality principles, and avoid using singular value decomposition of coefficient matrix. Thus, the reconstruction is geometric valid. Further more, we obtain the right characteristic matching points and accurate trifocal tensor with effective robust estimate.
We researched the stratified reconstruction, computed the infinity plane from projective reconstruction by solving the Cheirality inequality, and obtained the internal parameters matrix of camera by infinite homography. With camera self-calibration, projective reconstruction can be transform to affine reconstruction, and finally achieve metric correction. Compared to traditional multiple-view reconstruction methods, our method obtained an accurate minimum case solution, can be used wider as we reconstructed the 3D object without vanishing points or lines.
目前,射影重构过程中计算三焦点张量多采用系数矩阵奇异值分解方法,该方法容易引入数据冗余,导致解得的三焦点张量几何无效,对后期重建有一定影响。度量校正过程中,传统方法需要摄像机按照特定轨迹移动或者需要图像中具有消失点或平行线等信息,具有一定的应用局限性。针对重构过程中存在的这些问题,本文主要工作如下:
结合对偶化原则,采用基于最小配置实现三焦点张量的计算,从而可避免奇异值分解方法带来的数据冗余,得到几何有效的射影重构。同时结合自适应鲁棒估计和引导匹配算法,可有效去除误匹配干扰,获得较好的特征匹配点和精确的三焦点张量。
采用分层重构的方法,通过解Cheirality不等式计算射影重构中的无穷远平面,从而获得无穷单应,进而计算摄像机的内参数矩阵。在摄像机自标定的基础上将射影重构转换到仿射重构,最终实现度量校正。
与传统的多视图重建算法相比,本文提出的算法能够获得精确的最小配置解,重构时不需要借助消失点、消失线等优点,具有更广泛的应用前景。
Multiple-view reconstruction is extracting 3D information from multiple views of the object, and intent to reconstruction the object. Recent years, academic world has attached attention to Multiple-view reconstruction. This technology is widely used in the field of robot manufacturing, surface inspection, medical imaging, surgical navigation, image registration, and so on. Multiple-view reconstruction is based on multiple views, and can extract useful information from at least three images. The process can be divided into projective reconstruction and metric correction.
Currently, the projective reconstruction commonly uses singular value decomposition of coefficient matrix to compute the trifocal tensor. This method easily leads to geometric invalid solution as it takes data redundancy in. In the process of metric correction, traditional methods are limited in practical application as they need vanishing points, vanishing lines or the special movement of camera. In this thesis, our study and discuss focused on the above two problems.
We computed the trifocal tensor based on minimum case with duality principles, and avoid using singular value decomposition of coefficient matrix. Thus, the reconstruction is geometric valid. Further more, we obtain the right characteristic matching points and accurate trifocal tensor with effective robust estimate.
We researched the stratified reconstruction, computed the infinity plane from projective reconstruction by solving the Cheirality inequality, and obtained the internal parameters matrix of camera by infinite homography. With camera self-calibration, projective reconstruction can be transform to affine reconstruction, and finally achieve metric correction. Compared to traditional multiple-view reconstruction methods, our method obtained an accurate minimum case solution, can be used wider as we reconstructed the 3D object without vanishing points or lines.
引文
[1]张青,唐守正,李永慈,等.基于三焦点张量的树木图像匹配.计算机工程与应用,2005,41(31):198~199,217.
[2] Songkran Jarusirisawad, Hideo Saito. 3DTV view generation using uncalibrated pure rotating and zooming cameras. Signal Processing: Image Communication,2009,24:17~30.
[3] Andrew W. Fitzgibbon, Andrew Zisserman. Automatic camera recovery for closed or open image sequences. Proceedings of the 5th European Conference on Computer Vision,1998,1: 311~326.
[4]李海滨,郝向阳,山海涛,等.基于同一场景3张影像的相机自标定算法.测绘科学技术学报,2008,25(2):131~134.
[5] Chia-Ming Cheng, Shu-Fan Wang, Chin-Hung Teng, et al. Image-based three dimensional reconstruction for Chinese treasure-Jadeite Cabbage with Insects. Computers & Graphics,2008,32:682~694.
[6]程鸿,章权兵,周婷婷,等.基于无穷单应的线性多视图重构.计算机工程与应用,2005,41(34):57~58.
[7] Jian Yao, Wai-Kuen Cham. Robust multi-view feature matching from multiple unordered views. Pattern Recognition,2007,40(11):3081~3099.
[8]章权兵,王海贤,韦穗.线性多视图重构的新算法.中国图象图形学报,2004,9(10): 1210~1215.
[9] H.Christophter Longuet-Higgins. A computer algorithm for reconstructing a scence from two projections. Nature,1981,293:133~135.
[10] O.D.Faugeras, Q.T.Luong, S.J.Maybank. Camera self-calibration: Theory and experiments. European Conference on Computer Vision.1992,588:563~578.
[11] Richard I.Hartley, In defense of the eight-point algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1997, 19(6):580~593.
[12] Haifeng Chen, Peter Meer. Robust regression with projection based M-estimators. The 9th International Conference on Computer Vision. 2003, 2:878~885.
[13] Martin A.Fischler, Robert C.Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of ACM, 1981, 24(6):381~395.
[14] Etienne Vincent, Robert Laganiere. Models from image triplets using epipolar gradient features. Image and Vision Computing, 2007, 25(11):1699~1708.
[15]马颂德,张正友.计算机视觉-计算理论与算法基础,北京:科学出版社, 1998:52~71.
[16]舒远,谈正,丁礼儒.利用空间正交约束的相机自标定和三维重建.西安交通大学学报,2005,39(2):138~141.
[17]祝海江,吴福朝.基于一组对应消失线的度量重建.软件学报,2004,15(5):666~672.
[18]孙凤梅,吴福朝,胡占义.由平行平面的投影确定无穷远平面的单应矩阵.软件学报,2003,14(5):936~946.
[19]孟晓桥,胡占义.摄像机自标定方法的研究与进展.自动化学报,2003,29(1):110~124.
[20] A.Habed, B.Boufama. Camera self-calibration from bivariate polynomials derived from Kruppa's equations. Pattern Recognition,2008,41(8):2484~2492.
[21] Marc Pollefeys, Luc Van Gool. Self-calibration from the absolute conic on the plane at infinity. Computer Analysis of Images and Patterns,1997,1296:175~182.
[22] Marc Pollefeys, Luc Van Gool. Stratified self-calibration with the modulus constraint. IEEE Transactions on Pattern Analysis and Machine Intelligence,1999,21(8):707~724.
[23]彭亚丽,马光胜.基于模约束的自定标方法,航空计算技术,2007,37(1):1~3,7.
[24] Hartley Richard, Zisserman Andrew. Multiple View Geometry in Computer Vision. London:Cambridge University Press,2003:23~233.
[25] Quan-Tuan Luong, Olivier D.Faugeras. The fundamental matrix: Theory, algorithm, and stability analysis. International Journal Computer Vision,1996,17(1):43~75.
[26] P.H.S. Torr, A.Zisserman. Robust parameterization and computation of the trifocal tensor. Image and Vision Computing,1997,15(8):591~605.
[27] David Nister. Untwisting a projective reconstruction. International Journal of Computer Vision,2004:165~183.
[28] Manolis I.A Lourakis, Antonis A.Argyros. Efficient, causal camera tracking in unprepared environments. Computer Vision and Image Understanding,2005,99(2):259~290.
[29]王付新,黄毓瑜,孟偲,等.三维重建中特征点提取算法的研究与实现.工程图学学报,2007,3:91~96.
[30]朱如军,孙先仿.一种改进的角点检测方法.计算机应用与软件.2005, 22(1):105~106, 129.
[31] Zhang Zhengyou. Determining the epipolar geometry and its uncertainty: A review. International Journal Computer Vision,1998,27(2):161~198.
[32]李晓红,王若朴.基于航空序列影像的图像匹配.西安工业学院学报.2005, 25(6):555~557.
[33] Richard I.Hartley. Projective reconstruction and invariants from multiple images. IEEE Transactions on Pattern Analysis and Machine Intelligence,1994,16(10):1036~1041.
[34] Stefan Carlsson. Duality of reconstruction and positioning from projective views. IEEE Workshop on Representation of Visual Scenes,1995:85~92.
[35] D.Weinshall, M.Werman, A.Shashua. Shape descriptors: Bilinear, trilinear and quadrilinear relations for multi-point geometry and linear projective reconstruction algorithms. IEEE Workshop on Representation of Visual Scenes,1995:58~65.
[36]张光澄,王文娟,韩红蕾.非线性最优化计算方法.北京:高等教育出版社,2005:35~42.
[37]席靓,李海滨,丛佃伟.三焦点张量的最小代数误差非迭代算法研究.海军航空工程学院学报,2007,22(5):587~590.
[38] Wei Xu, Jane Mulliqan. Robust relative pose estimation with integrated cheirality constraint. 2008 19th International Conference on Pattern Recognition,2008:1~4.
[39] Tomas Werner, Tomas Pajdla. Cheirality in epipolar geometry. The IEEE International Conference on Computer Vision,2001,1:548~553.
[40]罗进,张志军.单纯形法中的迭代思想.湖北工业大学学报,2008,23(1):95~96.
[41]赵丽梅.基于OpenGL的三维物体的多视角显示.交通与计算机,2004, 22(3):115~117.
[2] Songkran Jarusirisawad, Hideo Saito. 3DTV view generation using uncalibrated pure rotating and zooming cameras. Signal Processing: Image Communication,2009,24:17~30.
[3] Andrew W. Fitzgibbon, Andrew Zisserman. Automatic camera recovery for closed or open image sequences. Proceedings of the 5th European Conference on Computer Vision,1998,1: 311~326.
[4]李海滨,郝向阳,山海涛,等.基于同一场景3张影像的相机自标定算法.测绘科学技术学报,2008,25(2):131~134.
[5] Chia-Ming Cheng, Shu-Fan Wang, Chin-Hung Teng, et al. Image-based three dimensional reconstruction for Chinese treasure-Jadeite Cabbage with Insects. Computers & Graphics,2008,32:682~694.
[6]程鸿,章权兵,周婷婷,等.基于无穷单应的线性多视图重构.计算机工程与应用,2005,41(34):57~58.
[7] Jian Yao, Wai-Kuen Cham. Robust multi-view feature matching from multiple unordered views. Pattern Recognition,2007,40(11):3081~3099.
[8]章权兵,王海贤,韦穗.线性多视图重构的新算法.中国图象图形学报,2004,9(10): 1210~1215.
[9] H.Christophter Longuet-Higgins. A computer algorithm for reconstructing a scence from two projections. Nature,1981,293:133~135.
[10] O.D.Faugeras, Q.T.Luong, S.J.Maybank. Camera self-calibration: Theory and experiments. European Conference on Computer Vision.1992,588:563~578.
[11] Richard I.Hartley, In defense of the eight-point algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1997, 19(6):580~593.
[12] Haifeng Chen, Peter Meer. Robust regression with projection based M-estimators. The 9th International Conference on Computer Vision. 2003, 2:878~885.
[13] Martin A.Fischler, Robert C.Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of ACM, 1981, 24(6):381~395.
[14] Etienne Vincent, Robert Laganiere. Models from image triplets using epipolar gradient features. Image and Vision Computing, 2007, 25(11):1699~1708.
[15]马颂德,张正友.计算机视觉-计算理论与算法基础,北京:科学出版社, 1998:52~71.
[16]舒远,谈正,丁礼儒.利用空间正交约束的相机自标定和三维重建.西安交通大学学报,2005,39(2):138~141.
[17]祝海江,吴福朝.基于一组对应消失线的度量重建.软件学报,2004,15(5):666~672.
[18]孙凤梅,吴福朝,胡占义.由平行平面的投影确定无穷远平面的单应矩阵.软件学报,2003,14(5):936~946.
[19]孟晓桥,胡占义.摄像机自标定方法的研究与进展.自动化学报,2003,29(1):110~124.
[20] A.Habed, B.Boufama. Camera self-calibration from bivariate polynomials derived from Kruppa's equations. Pattern Recognition,2008,41(8):2484~2492.
[21] Marc Pollefeys, Luc Van Gool. Self-calibration from the absolute conic on the plane at infinity. Computer Analysis of Images and Patterns,1997,1296:175~182.
[22] Marc Pollefeys, Luc Van Gool. Stratified self-calibration with the modulus constraint. IEEE Transactions on Pattern Analysis and Machine Intelligence,1999,21(8):707~724.
[23]彭亚丽,马光胜.基于模约束的自定标方法,航空计算技术,2007,37(1):1~3,7.
[24] Hartley Richard, Zisserman Andrew. Multiple View Geometry in Computer Vision. London:Cambridge University Press,2003:23~233.
[25] Quan-Tuan Luong, Olivier D.Faugeras. The fundamental matrix: Theory, algorithm, and stability analysis. International Journal Computer Vision,1996,17(1):43~75.
[26] P.H.S. Torr, A.Zisserman. Robust parameterization and computation of the trifocal tensor. Image and Vision Computing,1997,15(8):591~605.
[27] David Nister. Untwisting a projective reconstruction. International Journal of Computer Vision,2004:165~183.
[28] Manolis I.A Lourakis, Antonis A.Argyros. Efficient, causal camera tracking in unprepared environments. Computer Vision and Image Understanding,2005,99(2):259~290.
[29]王付新,黄毓瑜,孟偲,等.三维重建中特征点提取算法的研究与实现.工程图学学报,2007,3:91~96.
[30]朱如军,孙先仿.一种改进的角点检测方法.计算机应用与软件.2005, 22(1):105~106, 129.
[31] Zhang Zhengyou. Determining the epipolar geometry and its uncertainty: A review. International Journal Computer Vision,1998,27(2):161~198.
[32]李晓红,王若朴.基于航空序列影像的图像匹配.西安工业学院学报.2005, 25(6):555~557.
[33] Richard I.Hartley. Projective reconstruction and invariants from multiple images. IEEE Transactions on Pattern Analysis and Machine Intelligence,1994,16(10):1036~1041.
[34] Stefan Carlsson. Duality of reconstruction and positioning from projective views. IEEE Workshop on Representation of Visual Scenes,1995:85~92.
[35] D.Weinshall, M.Werman, A.Shashua. Shape descriptors: Bilinear, trilinear and quadrilinear relations for multi-point geometry and linear projective reconstruction algorithms. IEEE Workshop on Representation of Visual Scenes,1995:58~65.
[36]张光澄,王文娟,韩红蕾.非线性最优化计算方法.北京:高等教育出版社,2005:35~42.
[37]席靓,李海滨,丛佃伟.三焦点张量的最小代数误差非迭代算法研究.海军航空工程学院学报,2007,22(5):587~590.
[38] Wei Xu, Jane Mulliqan. Robust relative pose estimation with integrated cheirality constraint. 2008 19th International Conference on Pattern Recognition,2008:1~4.
[39] Tomas Werner, Tomas Pajdla. Cheirality in epipolar geometry. The IEEE International Conference on Computer Vision,2001,1:548~553.
[40]罗进,张志军.单纯形法中的迭代思想.湖北工业大学学报,2008,23(1):95~96.
[41]赵丽梅.基于OpenGL的三维物体的多视角显示.交通与计算机,2004, 22(3):115~117.