三维重建中特征提取与匹配技术的研究
|
摘要
近年来,三维重建技术在考古学、建筑学、地质学、虚拟现实、机器人导航、物体识别以及军事等领域得到越来越广泛的应用。同时,随着计算机技术及网络技术的迅速发展,利用计算机把图像或视频等二维结构信息重建出具有真实感的三维模型的三维重建技术也深入到人们生活的方方面面,成为当前新的研究热点。
特征提取与匹配是三维重建中至关重要的一步,是欧氏重建和摄像机自标定等计算机视觉技术的基础,其匹配精度直接影响后续重建的效果。本文重点研究了特征提取与匹配技术中涉及的两个主要问题,其一是提高特征点描述子的独特性的研究,其二是序列图像中误匹配点剔除策略的研究。
为了提高特征点描述子的独特性,本文提出一种基于圆环块邻域划分方式和规范化颜色照度不变量的描述子(CCD)。首先,在极坐标空间中采用圆环块结构划分特征点邻域,形成24个不重叠的子区域,并根据子区域与特征点的距离计算各子区域的权重;其次,根据各子区域内的规范化RGB三通道颜色强度差形成照度不变量;最后,结合权重与照度不变量,建立72维特征向量描述特征点。
针对序列图像场景中存在大量相似结构而致使一些误匹配产生的问题,本文提出了一种基于几何约束关系的误匹配点剔除算法。在相同尺度下根据匹配点对的运动方向一致和运动距离相等的关系来剔除误匹配;在不同尺度下根据匹配点对间连线相交的关系来剔除误匹配。
实验结果证明了在尺度缩放、视点变化、图像平移旋转等几何变换下,本文提出的相关算法均通过真实图像的实验加以验证,获得了满意的实验效果。
In recent years, three-dimensional reconstruction technology is widely used in some applications such as archeology, architecture, geology, virtual reality, robot navigation, object recognition and military fields. Meanwhile, with the rapid development of computer technology and computer network, the 3D reconstruction can be seen in all aspects of life and also has become the new hotpot, the 3D reconstruction is the process converting two-dimensional structure information such as the images or videos to real three-dimensional model by the computers.
Feature extraction and matching is the first and vital step of 3D reconstruction, and it is the base of Euclidean reconstruction and camera self-calibration etc. The precision of image matching directly affects the results of following steps in the whole process. This paper focuses on two problems of feature extraction and matching. One is the distinctiveness of descriptors, and the other is the strategy of mismatching elimination for sequence images.
To improve the distinctiveness of descriptor, in this paper, a new descriptor based on circle-blocks neighborhood division and normalized color illumination invariant (CCD) is presented. First, it adopts circle-blocks structure to divide neighborhood of feature points into 24 non-overlapping sub-regions in the polar coordinate space, and then gives weight calculation method for evaluating the effect of every sub-region according to the distance between every sub-region and the feature point. Next, the illumination invariant is formed by the normalized RGB triple channels color intensity differences. Finally, a 72-dimensional feature vector will be established by combining the weights with illumination invariant.
To solve the problem of mismatching caused by the large number of similar structures in sequence images, this paper presents a mismatching points elimination algorithm based on the geometric constraint conditions for sequence images. The mismatching points are eliminated according to the relationship of consistent motion orientations and equivalent motion distances between them in the same scale. And the mismatching points are eliminated according to the intersection relationship of lines between them in different scales.
Experimental results on real images demonstrated that the new algorithms proposed in this paper are valid and robust under various geometric and photometric transformations application such as scale changes, viewpoint changes, translation transforms etc.
特征提取与匹配是三维重建中至关重要的一步,是欧氏重建和摄像机自标定等计算机视觉技术的基础,其匹配精度直接影响后续重建的效果。本文重点研究了特征提取与匹配技术中涉及的两个主要问题,其一是提高特征点描述子的独特性的研究,其二是序列图像中误匹配点剔除策略的研究。
为了提高特征点描述子的独特性,本文提出一种基于圆环块邻域划分方式和规范化颜色照度不变量的描述子(CCD)。首先,在极坐标空间中采用圆环块结构划分特征点邻域,形成24个不重叠的子区域,并根据子区域与特征点的距离计算各子区域的权重;其次,根据各子区域内的规范化RGB三通道颜色强度差形成照度不变量;最后,结合权重与照度不变量,建立72维特征向量描述特征点。
针对序列图像场景中存在大量相似结构而致使一些误匹配产生的问题,本文提出了一种基于几何约束关系的误匹配点剔除算法。在相同尺度下根据匹配点对的运动方向一致和运动距离相等的关系来剔除误匹配;在不同尺度下根据匹配点对间连线相交的关系来剔除误匹配。
实验结果证明了在尺度缩放、视点变化、图像平移旋转等几何变换下,本文提出的相关算法均通过真实图像的实验加以验证,获得了满意的实验效果。
In recent years, three-dimensional reconstruction technology is widely used in some applications such as archeology, architecture, geology, virtual reality, robot navigation, object recognition and military fields. Meanwhile, with the rapid development of computer technology and computer network, the 3D reconstruction can be seen in all aspects of life and also has become the new hotpot, the 3D reconstruction is the process converting two-dimensional structure information such as the images or videos to real three-dimensional model by the computers.
Feature extraction and matching is the first and vital step of 3D reconstruction, and it is the base of Euclidean reconstruction and camera self-calibration etc. The precision of image matching directly affects the results of following steps in the whole process. This paper focuses on two problems of feature extraction and matching. One is the distinctiveness of descriptors, and the other is the strategy of mismatching elimination for sequence images.
To improve the distinctiveness of descriptor, in this paper, a new descriptor based on circle-blocks neighborhood division and normalized color illumination invariant (CCD) is presented. First, it adopts circle-blocks structure to divide neighborhood of feature points into 24 non-overlapping sub-regions in the polar coordinate space, and then gives weight calculation method for evaluating the effect of every sub-region according to the distance between every sub-region and the feature point. Next, the illumination invariant is formed by the normalized RGB triple channels color intensity differences. Finally, a 72-dimensional feature vector will be established by combining the weights with illumination invariant.
To solve the problem of mismatching caused by the large number of similar structures in sequence images, this paper presents a mismatching points elimination algorithm based on the geometric constraint conditions for sequence images. The mismatching points are eliminated according to the relationship of consistent motion orientations and equivalent motion distances between them in the same scale. And the mismatching points are eliminated according to the intersection relationship of lines between them in different scales.
Experimental results on real images demonstrated that the new algorithms proposed in this paper are valid and robust under various geometric and photometric transformations application such as scale changes, viewpoint changes, translation transforms etc.
引文
[1] Harris C, Stephens M J. A combined corner and edge detector [J]. In Alvey Vision Conference Manchester, 1988:147-152
[2] Schmid C, Mohr R. Local Gray value Invariants for Image Retrieval [J]. IEEE Transactions On Pattern Analysis and Machine Intelligence, 1997, V19(5):530-535
[3] Schmid, C. Mohr and Bauckhage. Evaluation of interest point detectors [J]. International Journal of Computer Vision, 2000, V37(2):151–172
[4] K. Mikolajczyk and C. Schmid. Scale and Affine Invariant Interest Point Detectors [J]. International Journal of Computer Vision, 2004, V1(60):63-86
[5] Montesinos P, Gouet V, DericheR. Differential invariants for color images[C]. International Conference on Pattern Recognition( ICPR), 1998:838-840
[6] P.Beardsley, P.Tort, and A.Zisserman. 3D model acquisition from extended image sequences[C]. In Proceedings of the 4th European Conference on Computer vision, LNCS 1065, Cambridge, 1996:683-695
[7] P.Pritehett and A.Zisserman. Matching and Reconstruction from Widely Separated Views [J]. Lecture Notes in Computer Science,Springer-Verlag,1998, (1506):78-92
[8] M.Lhuillier and Q.T.Luong, Robust Dense Matching Using Local and Global Geometric Constraints[C]. In Proceedings of the 16th International Conference on Pattern Recognition, Barcelona, Spain,2000,(1):968-972
[9] Cheng Y Q, Wang X G. Three-dimensional reconstruction of points and lines with unknown correspondence across Images [J]. International Journal of Computer Vision,2001, V45(2):129-156
[10] D. Lowe. Distinctive Image Features from Scale-Invariant Key-points [J]. International Journal of Computer Vision,2004, V2(60):91-110
[11] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool. Speeded-up robust features (SURF) [J]. International Journal on Computer Vision and Image Understanding,2008, V110(3):346–359
[12] Y. Ke, R.Sukthankar. PCA-SIFT: a more distinctive representation for local image descriptors[C]. in Proceedings of Computer Vision and Pattern Recognition (CVPR),2004, (1): 506-513
[13] K. Mikolajczyk and C. Schmid. A Performance Evaluation of Local Descriptors [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2005, V27(10):1615–1630
[14] S. Belongie, J. Malik, and J. Puzicha. Shape Matching and Object Recognition Using Shape Contexts [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2002, V2(4):509-522
[15] Huang C R, Chen C R, Chung P C. Contrast context histogram-an efficient discriminating local descriptor for object recognition and image matching[C]. in Proceedings of Computer Vision and Pattern Recognition,2008, V41(10):3071-3077
[16] Yuehua Tao, Youming Xia, Tianwei Xu, xiaoxiao Chi. Research Progress of the Scale Invariant Feature Transform (SIFT) Descriptors [J]. Journal of Convergence Information,2010, V5(1):117?121
[17] T. Tuytelaars and K. Mikolajczyk. Local invariant feature detectors: A survey [J]. Foundations and Trends in Computer Graphics and Vision,2008, V3(3):177–280
[18] Fishier M A, Boles R C. Random Sample Consensus: A Paradigm for Model Fitting with Applicationsto Image Analysis and Automated Cartography[J].Comm Assoc Comp Mach, 1981, V24(6):381-395
[19] Richard I. Hartley: In Defense of the Eight-Point Algorithm [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI),1997, V19(6):580-593
[20] Huang. J, Lai. S, Cheng. C. Robust Fundamental Matrix Estimation with Accurate Outlier Detection [J]. J. Inf. Sci. Eng(JISE),2007, V23(4):1213-1225
[21] Marc Pollefeys, Luc J Van Gool, Marten Vergauwen. Visual Modeling with a Hand-Held Camera [J]. International Journal of Computer Vision,2004, V59(3):207-232
[22] G. J. Burghouts and J.-M. Geusebroek. Performance evaluation of local color invariants [J]. Computer Vision and Image Understanding,2009, V113(1):48–62
[23]尹平,王润生.自适应多尺度边缘检测[J].软件学报,2000,11(8):990～994
[24]季虎,孙即祥,邵晓芳,毛玲.图像边缘提取方法及展望[J].计算机工程与应用,2004,40(14):70-74
[25]邵平,杨路明.基于模板分解和积分图像的快速Kirsch边缘检测[J].自动化学报,2007,33(8):795-800
[26]张翔,刘媚洁,陈立伟.基于数学形态学的边缘提取方法[J].电子科技大学学报,2002,31(5):492-495
[27]乔警卫,胡少兴.三维重建特征点提取与匹配算法研究[J].系统仿真学报,2008,20:400-403
[28]姜璐璐,彭健.基于极限几何约束的非定标图像的立体匹配[J].计算机应用,2007,27(11):2800-2803
[29]宰小涛,赵宇明.基于SIFT特征描述子的立体匹配算法[J].微计算机信息,2007,23(8-3):285-287
[30]魏志强,黄磊,纪筱鹏.基于点特征的序列图像匹配方法研究[J].中国图象图形学报,2009,14(3):525-530
[31]马颂德.计算机视觉——计算理论与算法基础[M].北京:科学出版社,1997
[32]余萍,袁辉,赵振兵,王碧翠.图像识别中的兴趣点匹配方法研究[J].计算机工程与应用,2010,46(5):132~135
[33]周兵,沈钧毅,彭勤科.一种新的基于对称色彩空域特征的图像匹配方法[J].小型微型计算机系统,2005,26(1):147-150
[34]李强,张钹.一种基于图像灰度的快速匹配算法[J].软件学报,2006,17(2):216-222
[35]杨晓敏,吴炜,卿粼波,华骅,何小海.图像特征点提取及匹配技术[J].光学精密工程,2009,17(9):2276-2282
[36]纪华,吴元昊,孙宏海,王延杰.结合全局信息的SIFT特征匹配算法[J].光学精密工程, 2009,17(2):439-444
[37]张春美,龚志辉.改进SIFT特征在图像匹配中的应用[J].计算机工程与应用,2008,44(2):95-98
[38]陈泽志,吴成柯,刘勇.基础矩阵估计的加权归一化线性算法[J].软件学报,2001,12(3):420-426
[39]郭继东,向辉.一个基本矩阵的鲁棒估计算法[J].计算机应用,2005,25(12):2845-2848
[40]姜璐璐,彭健.基于极限几何约束的非定标图像的立体匹配[J].计算机应用,2007,27(11):2800-2803
[41]梁栋,童强,屈磊.一种基于极几何和单应约束的图像匹配算法[J].系统仿真学报,2006,18(1):44-46
[42]黄锡山,陈慧津,陈哲.景像匹配误匹配点的剔除算法[J].中国图象图形学报,2002,7 (8):783-787
[43]张洁玉,白小晶,徐丽燕.基于空间分布描述符的SIFT误匹配校正方法[J].中国图象图形学报,2009,14(7):1369-1377
[44]鲍文霞,梁栋,程志友,宣善立,王年.一种基于图理论的非刚体形状匹配算法[J].仪器仪表学报,2009,30(10):2027-2032
[45]陈燕军,郑江滨,史文波,李秀秀.一种运动捕获系统中误匹配点的剔除算法[J].计算机应用研究,2009,26(2):422-424
[2] Schmid C, Mohr R. Local Gray value Invariants for Image Retrieval [J]. IEEE Transactions On Pattern Analysis and Machine Intelligence, 1997, V19(5):530-535
[3] Schmid, C. Mohr and Bauckhage. Evaluation of interest point detectors [J]. International Journal of Computer Vision, 2000, V37(2):151–172
[4] K. Mikolajczyk and C. Schmid. Scale and Affine Invariant Interest Point Detectors [J]. International Journal of Computer Vision, 2004, V1(60):63-86
[5] Montesinos P, Gouet V, DericheR. Differential invariants for color images[C]. International Conference on Pattern Recognition( ICPR), 1998:838-840
[6] P.Beardsley, P.Tort, and A.Zisserman. 3D model acquisition from extended image sequences[C]. In Proceedings of the 4th European Conference on Computer vision, LNCS 1065, Cambridge, 1996:683-695
[7] P.Pritehett and A.Zisserman. Matching and Reconstruction from Widely Separated Views [J]. Lecture Notes in Computer Science,Springer-Verlag,1998, (1506):78-92
[8] M.Lhuillier and Q.T.Luong, Robust Dense Matching Using Local and Global Geometric Constraints[C]. In Proceedings of the 16th International Conference on Pattern Recognition, Barcelona, Spain,2000,(1):968-972
[9] Cheng Y Q, Wang X G. Three-dimensional reconstruction of points and lines with unknown correspondence across Images [J]. International Journal of Computer Vision,2001, V45(2):129-156
[10] D. Lowe. Distinctive Image Features from Scale-Invariant Key-points [J]. International Journal of Computer Vision,2004, V2(60):91-110
[11] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool. Speeded-up robust features (SURF) [J]. International Journal on Computer Vision and Image Understanding,2008, V110(3):346–359
[12] Y. Ke, R.Sukthankar. PCA-SIFT: a more distinctive representation for local image descriptors[C]. in Proceedings of Computer Vision and Pattern Recognition (CVPR),2004, (1): 506-513
[13] K. Mikolajczyk and C. Schmid. A Performance Evaluation of Local Descriptors [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2005, V27(10):1615–1630
[14] S. Belongie, J. Malik, and J. Puzicha. Shape Matching and Object Recognition Using Shape Contexts [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2002, V2(4):509-522
[15] Huang C R, Chen C R, Chung P C. Contrast context histogram-an efficient discriminating local descriptor for object recognition and image matching[C]. in Proceedings of Computer Vision and Pattern Recognition,2008, V41(10):3071-3077
[16] Yuehua Tao, Youming Xia, Tianwei Xu, xiaoxiao Chi. Research Progress of the Scale Invariant Feature Transform (SIFT) Descriptors [J]. Journal of Convergence Information,2010, V5(1):117?121
[17] T. Tuytelaars and K. Mikolajczyk. Local invariant feature detectors: A survey [J]. Foundations and Trends in Computer Graphics and Vision,2008, V3(3):177–280
[18] Fishier M A, Boles R C. Random Sample Consensus: A Paradigm for Model Fitting with Applicationsto Image Analysis and Automated Cartography[J].Comm Assoc Comp Mach, 1981, V24(6):381-395
[19] Richard I. Hartley: In Defense of the Eight-Point Algorithm [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI),1997, V19(6):580-593
[20] Huang. J, Lai. S, Cheng. C. Robust Fundamental Matrix Estimation with Accurate Outlier Detection [J]. J. Inf. Sci. Eng(JISE),2007, V23(4):1213-1225
[21] Marc Pollefeys, Luc J Van Gool, Marten Vergauwen. Visual Modeling with a Hand-Held Camera [J]. International Journal of Computer Vision,2004, V59(3):207-232
[22] G. J. Burghouts and J.-M. Geusebroek. Performance evaluation of local color invariants [J]. Computer Vision and Image Understanding,2009, V113(1):48–62
[23]尹平,王润生.自适应多尺度边缘检测[J].软件学报,2000,11(8):990～994
[24]季虎,孙即祥,邵晓芳,毛玲.图像边缘提取方法及展望[J].计算机工程与应用,2004,40(14):70-74
[25]邵平,杨路明.基于模板分解和积分图像的快速Kirsch边缘检测[J].自动化学报,2007,33(8):795-800
[26]张翔,刘媚洁,陈立伟.基于数学形态学的边缘提取方法[J].电子科技大学学报,2002,31(5):492-495
[27]乔警卫,胡少兴.三维重建特征点提取与匹配算法研究[J].系统仿真学报,2008,20:400-403
[28]姜璐璐,彭健.基于极限几何约束的非定标图像的立体匹配[J].计算机应用,2007,27(11):2800-2803
[29]宰小涛,赵宇明.基于SIFT特征描述子的立体匹配算法[J].微计算机信息,2007,23(8-3):285-287
[30]魏志强,黄磊,纪筱鹏.基于点特征的序列图像匹配方法研究[J].中国图象图形学报,2009,14(3):525-530
[31]马颂德.计算机视觉——计算理论与算法基础[M].北京:科学出版社,1997
[32]余萍,袁辉,赵振兵,王碧翠.图像识别中的兴趣点匹配方法研究[J].计算机工程与应用,2010,46(5):132~135
[33]周兵,沈钧毅,彭勤科.一种新的基于对称色彩空域特征的图像匹配方法[J].小型微型计算机系统,2005,26(1):147-150
[34]李强,张钹.一种基于图像灰度的快速匹配算法[J].软件学报,2006,17(2):216-222
[35]杨晓敏,吴炜,卿粼波,华骅,何小海.图像特征点提取及匹配技术[J].光学精密工程,2009,17(9):2276-2282
[36]纪华,吴元昊,孙宏海,王延杰.结合全局信息的SIFT特征匹配算法[J].光学精密工程, 2009,17(2):439-444
[37]张春美,龚志辉.改进SIFT特征在图像匹配中的应用[J].计算机工程与应用,2008,44(2):95-98
[38]陈泽志,吴成柯,刘勇.基础矩阵估计的加权归一化线性算法[J].软件学报,2001,12(3):420-426
[39]郭继东,向辉.一个基本矩阵的鲁棒估计算法[J].计算机应用,2005,25(12):2845-2848
[40]姜璐璐,彭健.基于极限几何约束的非定标图像的立体匹配[J].计算机应用,2007,27(11):2800-2803
[41]梁栋,童强,屈磊.一种基于极几何和单应约束的图像匹配算法[J].系统仿真学报,2006,18(1):44-46
[42]黄锡山,陈慧津,陈哲.景像匹配误匹配点的剔除算法[J].中国图象图形学报,2002,7 (8):783-787
[43]张洁玉,白小晶,徐丽燕.基于空间分布描述符的SIFT误匹配校正方法[J].中国图象图形学报,2009,14(7):1369-1377
[44]鲍文霞,梁栋,程志友,宣善立,王年.一种基于图理论的非刚体形状匹配算法[J].仪器仪表学报,2009,30(10):2027-2032
[45]陈燕军,郑江滨,史文波,李秀秀.一种运动捕获系统中误匹配点的剔除算法[J].计算机应用研究,2009,26(2):422-424