局部灰度极值模式及其局部描述符
|
摘要
目的近年来,由于局部图像描述符在大的视角与光度变化、噪声、局部遮挡等方面具有良好性能,已成功应用于图像搜索、机器人导航、图像分类、视频行为识别等各种计算机视觉研究领域。方法提出了一种新的用于图像区域描述的局部特征:局部灰度极值模式(LIEP)。在离一个像素点半径不同的两个同心圆上分别均匀抽样相同点数的采样点,不同同心圆上采样点与中心像素点之间的夹角相互内插,分别独立计算每个同心圆上采样点的最大和最小灰度模式。计算半径小的同心圆上的最大灰度模式和半径大的同心圆上的最小灰度模式的2维联合分布,得到一种极值模式。再计算半径小的同心圆上的最小灰度模式和半径大的同心圆上的最大灰度模式的2维联合分布,得到另一种极值模式。最后对这2种极值模式进行级联,得到LIEP。相对于局部灰度序模式和局部二进制模式,LIEP在图像光度和几何变化下更稳定,抗噪声性能更强,出现模式错误的概率更小。LIEP在局部旋转不变坐标系统下计算,采用多支撑域和图像块全局灰度序空间汇聚方法得到一种新的局部图像描述符:LIEP空间分布直方图(LIEPH)。LIEPH描述符具有单调光照不变性和在不计算图像块主方向条件下保持旋转不变性。结果在标准图像匹配数据库上的实验表明:LIEPH的查全率-查错率曲线都位于最上方,匹配性能大大优于单支撑域描述符SIFT(scale invariant feature transform)、CS-LBP(center-symmetric local binary pattern)、LIOP(local intensity order pattern)、HRI-CSLTP(histogram of relative intensities and center-symmetric local ternary patterns)、EOD(exact order based descriptor)及多支撑域描述符MRRID(multisupport region rotation and intensity monotonic imariant descriptor)。在大的图像几何畸变下,LIEPH更能展现优越的匹配性能。在对描述符进行定量分析的实验中,当查错率(1-precision-sion)取固定值0. 4时,LIEPH描述符的查全率(recall)值在各种图像畸变下都是最大的。在标准图像匹配数据库上添加高斯和椒盐噪声的实验中,LIEPH的匹配性能远远优于MRRID。LIEPH算法的复杂度更低,计算时间接近MRRID的1/2。结论 LIEPH对局部图像区域的纹理统计特性具有很高的描述能力,在辨别性、鲁棒性和抗噪声方面的优越性能使其可以应用于复杂条件下的图像区域描述和匹配场合。
Objective Local image descriptors have been successfully applied to computer vision research,such as image search,robot navigation,image classification,and video action recognition. Local image descriptors perform effectively in the large viewpoint change of cameras,photometric change,noise,and local occlusion. Method A new local feature forimage region description,namely,local intensity extremum pattern( LIEP),is proposed in this study. The same number of pixel points is uniformly sampled on two concentric circles with different radii from one pixel point. The angles between the sampling points and the center pixel on different concentric circles interpolate each another. The maximum and minimum intensity patterns of each concentric circle are calculated independently. Two-dimensional joint distributions of the minimum intensity pattern on the concentric circle with a small radius and the maximum intensity pattern on the concentric circle with a large radius are computed. Subsequently,an intensity extreme pattern is obtained. Two-dimensional joint distributions of the maximum intensity pattern on the concentric circle with a small radius and the minimum intensity pattern on the concentric circle with a large radius are computed,and another intensity extreme pattern is obtained. The two extreme patterns are cascaded to obtain LIEP. From the calculation process of LIEP,the two LIEP sub-patterns will not change because the position of the maximum intensity pixel on a concentric circle and the position of the minimum intensity pixel on the other concentric circle do not change. Local patterns related to LIEP include local binary and intensity order patterns. Local binary pattern calculates the symbol of intensity difference between multiple pairs of pixels at the same time. Change in the symbol of intensity differences between any pairs of pixels will change the local binary pattern. Local intensity order pattern calculates the ranking of intensity value of multiple pixels and the change of the ranking of intensity value of any pixel will subsequently change the local intensity order pattern. Under the condition of image patch with adding Gaussian noise,the statistical histogram crossover of the LIEP feature between the origin image patch and image patch with adding Gaussian noise is higher than that of the local binary and intensity order patterns. Compared with local intensity order and binary patterns,the LIEP is more stable and robust to noise and image changes,and has smaller probability of pattern errors. LIEP is calculated in the local rotation-invariant coordinate system. A new local image descriptor,namely,LIEP histogram( LIEPH),is obtained using multiple support regions and the space convergence method of global intensity order in an image patch. The LIEPH descriptor has monotonous illumination invariance and keeps the rotation invariance without calculating the main direction of the image patch. Results Comparison experiments with other popular local descriptors were conducted out on the standard image matching database. Experiments show that MRRID and LIEPH are superior to SIFT,LIOP,CS-LBP,HRI-CSLTP,and EOD in all image distortion conditions. LIEPH is superior to MRRID under Boat 1-5 and Wall 1-5 image distortion; LIEPH is slightly superior under Graffiti 1-5,Boat 1-3,Wall 1-3,and UBC 1-5 image distortion.The matching performance of LIEPH is equivalent to that of MRRID under other image distortions. In other words,the matching performance of LIEPH is equal to,slightly,or much higher than that of MRRID under all image distortion conditions. Based on all above situations,LIEPH performs better than that of MRRID,and the matching performance of LIEPH is better in large image photometric and geometric distortions. Therefore,LIEPH has strong discrimination and robustness.The robustness in resisting large image geometric distortion of LIEPH is better than MRRID as well. In the quantitative analysis experiments of the descriptors,when 1-precision is 0. 4,the recall value of the LIEPH descriptor is largest under all types of image distortion. In the experiments of adding Gauss and salt-and-pepper noise to the standard image matching databases,the matching performance of LIEPH is better than that of MRRID. The algorithm complexity of LIEPH is lower,which is close to half of that of MRRID. Conclusion LIEPH has high capability to describe texture statistics in local image regions. The superior performance in discriminative power,robustness,and anti-noise enable the application of LIEPH to image region description and matching occasions under complex conditions.
Objective Local image descriptors have been successfully applied to computer vision research,such as image search,robot navigation,image classification,and video action recognition. Local image descriptors perform effectively in the large viewpoint change of cameras,photometric change,noise,and local occlusion. Method A new local feature forimage region description,namely,local intensity extremum pattern( LIEP),is proposed in this study. The same number of pixel points is uniformly sampled on two concentric circles with different radii from one pixel point. The angles between the sampling points and the center pixel on different concentric circles interpolate each another. The maximum and minimum intensity patterns of each concentric circle are calculated independently. Two-dimensional joint distributions of the minimum intensity pattern on the concentric circle with a small radius and the maximum intensity pattern on the concentric circle with a large radius are computed. Subsequently,an intensity extreme pattern is obtained. Two-dimensional joint distributions of the maximum intensity pattern on the concentric circle with a small radius and the minimum intensity pattern on the concentric circle with a large radius are computed,and another intensity extreme pattern is obtained. The two extreme patterns are cascaded to obtain LIEP. From the calculation process of LIEP,the two LIEP sub-patterns will not change because the position of the maximum intensity pixel on a concentric circle and the position of the minimum intensity pixel on the other concentric circle do not change. Local patterns related to LIEP include local binary and intensity order patterns. Local binary pattern calculates the symbol of intensity difference between multiple pairs of pixels at the same time. Change in the symbol of intensity differences between any pairs of pixels will change the local binary pattern. Local intensity order pattern calculates the ranking of intensity value of multiple pixels and the change of the ranking of intensity value of any pixel will subsequently change the local intensity order pattern. Under the condition of image patch with adding Gaussian noise,the statistical histogram crossover of the LIEP feature between the origin image patch and image patch with adding Gaussian noise is higher than that of the local binary and intensity order patterns. Compared with local intensity order and binary patterns,the LIEP is more stable and robust to noise and image changes,and has smaller probability of pattern errors. LIEP is calculated in the local rotation-invariant coordinate system. A new local image descriptor,namely,LIEP histogram( LIEPH),is obtained using multiple support regions and the space convergence method of global intensity order in an image patch. The LIEPH descriptor has monotonous illumination invariance and keeps the rotation invariance without calculating the main direction of the image patch. Results Comparison experiments with other popular local descriptors were conducted out on the standard image matching database. Experiments show that MRRID and LIEPH are superior to SIFT,LIOP,CS-LBP,HRI-CSLTP,and EOD in all image distortion conditions. LIEPH is superior to MRRID under Boat 1-5 and Wall 1-5 image distortion; LIEPH is slightly superior under Graffiti 1-5,Boat 1-3,Wall 1-3,and UBC 1-5 image distortion.The matching performance of LIEPH is equivalent to that of MRRID under other image distortions. In other words,the matching performance of LIEPH is equal to,slightly,or much higher than that of MRRID under all image distortion conditions. Based on all above situations,LIEPH performs better than that of MRRID,and the matching performance of LIEPH is better in large image photometric and geometric distortions. Therefore,LIEPH has strong discrimination and robustness.The robustness in resisting large image geometric distortion of LIEPH is better than MRRID as well. In the quantitative analysis experiments of the descriptors,when 1-precision is 0. 4,the recall value of the LIEPH descriptor is largest under all types of image distortion. In the experiments of adding Gauss and salt-and-pepper noise to the standard image matching databases,the matching performance of LIEPH is better than that of MRRID. The algorithm complexity of LIEPH is lower,which is close to half of that of MRRID. Conclusion LIEPH has high capability to describe texture statistics in local image regions. The superior performance in discriminative power,robustness,and anti-noise enable the application of LIEPH to image region description and matching occasions under complex conditions.
引文
[1]Xu Y X,Chen F.Recent advances in local image descriptor[J].Journal of Image and Graphics,2015,20(9):1133-1150.[许允喜,陈方.局部图像描述符最新研究进展[J].中国图象图形学报,2015,20(9):1133-1150.][DOI:10.11834/jig.20150901]
[2]Zhao W L,Ngo C W,Wang H Z.Fast covariant VLAD for image search[J].IEEE Transactions on Multimedia,2016,18(9):1843-1854.[DOI:10.1109/TMM.2016.2585023]
[3]Angeli A,Filliat D,Doncieux S,et al.Fast and incremental method for loop-closure detection using bags of visual words[J].IEEE Transactions on Robotics,2008,24(5):1027-1037.[DOI:10.1109/TRO.2008.2004514]
[4]Koniusz P,Yan F,Gosselin P H,et al.Higher-order occurrence pooling for bags-of-words:visual concept detection[J].IEEETransactions on Pattern Analysis and Machine Intelligence,2017,39(2):313-326.[DOI:10.1109/TPAMI.2016.2545667]
[5]Zhen X T,Zheng F,Shao L,et al.Supervised local descriptor learning for human action recognition[J].IEEE Transactions on Multimedia,2017,19(9):2056-2065.[DOI:10.1109/TMM.2017.2700204]
[6]Lowe D G.Distinctive image features from scale-invariant keypoints[J].International Journal of Computer Vision,2004,60(2):91-110.[DOI:10.1023/B:VISI.0000029664.99615.94]
[7]Bay H,Ess A,Tuytelaars T,et al.Speeded-up robust features(SURF)[J].Computer Vision and Image Understanding,2008,110(3):346-359.[DOI:10.1016/j.cviu.2007.09.014]
[8]Ambai M,Yoshida Y.CARD:compact and real-time descriptors[C]//Proceedings of 2011 IEEE International Conference on Computer Vision.Barcelona,Spain:IEEE,2011:97-104.[DOI:10.1109/ICCV.2011.6126230]
[9]Strecha C,Bronstein A,Bronstein M,et al.LDAHash:improved matching with smaller descriptors[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(1):66-78.[DOI:10.1109/TPAMI.2011.103]
[10]Trzcinski T,Christoudias M,Lepetit V.Learning image descriptors with boosting[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(3):597-610.[DOI:10.1109/TPAMI.2014.2343961]
[11]Simonyan K,Vedaldi A,Zisserman A.Learning local feature descriptors using convex optimisation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(8):1573-1585.[DOI:10.1109/TPAMI.2014.2301163]
[12]Huang D,Zhu C,Wang Y H,et al.HSOG:a novel local image descriptor based on histograms of the second-order gradients[J].IEEE Transactions on Image Processing,2014,23(11):4680-4695.[DOI:10.1109/TIP.2014.2353814]
[13]Chen J,Shan S G,He C,et al.WLD:a robust local image descriptor[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,32(9):1705-1720.[DOI:10.1109/TPA-MI.2009.155]
[14]Heikkila M,Pietikainen M,Schmid C.Description of interest regions with local binary patterns[J].Pattern Recognition,2009,42(3):425-436.[DOI:10.1016/j.patcog.2008.08.014]
[15]Tang F,Lim S H,Chang N L,et al.A novel feature descriptor invariant to complex brightness changes[C]//Proceedings of2009 IEEE Conference on Computer Vision and Pattern Recognition.Miami,USA:IEEE,2009:2631-2638.[DOI:10.1109/CVPR.2009.5206550]
[16]Gupta R,Patil H,Mittal A.Robust order-based methods for feature description[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Francisco,CA:IEEE,2010:334-341.[DOI:10.1109/CVPR.2010.5540195]
[17]Kim B,Yoo H,Sohn K.Exact order based feature descriptor for illumination robust image matching[J].Pattern Recognition,2013,46(12):3268-3278.[DOI:10.1016/j.patcog.2013.04.015]
[18]Fan B,Wu F C,Hu Z Y.Rotationally invariant descriptors using intensity order pooling[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(10):2031-2045.[DOI:10.1109/TPAMI.2011.277]
[19]Wang Z H,Fan B,Wu F C.Local intensity order pattern for feature description[C]//Proceedings of 2011 IEEE International Conference on Computer Vision.Barcelona,Spain:IEEE,2011:603-610.[DOI:10.1109/ICCV.2011.6126294]
[20]Mikolajczyk K,Tuytelaars T,Schmid C,et al.A comparison of affine region detectors[J].International Journal of Computer Vision,2005,65(1-2):43-72.[DOI:10.1007/s11263-005-3848-x]
[21]Mikolajczyk K,Schmid C.A performance evaluation of local descriptors[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2005,27(10):1615-1630.[DOI:10.1109/TPAMI.2005.188]
[2]Zhao W L,Ngo C W,Wang H Z.Fast covariant VLAD for image search[J].IEEE Transactions on Multimedia,2016,18(9):1843-1854.[DOI:10.1109/TMM.2016.2585023]
[3]Angeli A,Filliat D,Doncieux S,et al.Fast and incremental method for loop-closure detection using bags of visual words[J].IEEE Transactions on Robotics,2008,24(5):1027-1037.[DOI:10.1109/TRO.2008.2004514]
[4]Koniusz P,Yan F,Gosselin P H,et al.Higher-order occurrence pooling for bags-of-words:visual concept detection[J].IEEETransactions on Pattern Analysis and Machine Intelligence,2017,39(2):313-326.[DOI:10.1109/TPAMI.2016.2545667]
[5]Zhen X T,Zheng F,Shao L,et al.Supervised local descriptor learning for human action recognition[J].IEEE Transactions on Multimedia,2017,19(9):2056-2065.[DOI:10.1109/TMM.2017.2700204]
[6]Lowe D G.Distinctive image features from scale-invariant keypoints[J].International Journal of Computer Vision,2004,60(2):91-110.[DOI:10.1023/B:VISI.0000029664.99615.94]
[7]Bay H,Ess A,Tuytelaars T,et al.Speeded-up robust features(SURF)[J].Computer Vision and Image Understanding,2008,110(3):346-359.[DOI:10.1016/j.cviu.2007.09.014]
[8]Ambai M,Yoshida Y.CARD:compact and real-time descriptors[C]//Proceedings of 2011 IEEE International Conference on Computer Vision.Barcelona,Spain:IEEE,2011:97-104.[DOI:10.1109/ICCV.2011.6126230]
[9]Strecha C,Bronstein A,Bronstein M,et al.LDAHash:improved matching with smaller descriptors[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(1):66-78.[DOI:10.1109/TPAMI.2011.103]
[10]Trzcinski T,Christoudias M,Lepetit V.Learning image descriptors with boosting[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(3):597-610.[DOI:10.1109/TPAMI.2014.2343961]
[11]Simonyan K,Vedaldi A,Zisserman A.Learning local feature descriptors using convex optimisation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(8):1573-1585.[DOI:10.1109/TPAMI.2014.2301163]
[12]Huang D,Zhu C,Wang Y H,et al.HSOG:a novel local image descriptor based on histograms of the second-order gradients[J].IEEE Transactions on Image Processing,2014,23(11):4680-4695.[DOI:10.1109/TIP.2014.2353814]
[13]Chen J,Shan S G,He C,et al.WLD:a robust local image descriptor[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,32(9):1705-1720.[DOI:10.1109/TPA-MI.2009.155]
[14]Heikkila M,Pietikainen M,Schmid C.Description of interest regions with local binary patterns[J].Pattern Recognition,2009,42(3):425-436.[DOI:10.1016/j.patcog.2008.08.014]
[15]Tang F,Lim S H,Chang N L,et al.A novel feature descriptor invariant to complex brightness changes[C]//Proceedings of2009 IEEE Conference on Computer Vision and Pattern Recognition.Miami,USA:IEEE,2009:2631-2638.[DOI:10.1109/CVPR.2009.5206550]
[16]Gupta R,Patil H,Mittal A.Robust order-based methods for feature description[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Francisco,CA:IEEE,2010:334-341.[DOI:10.1109/CVPR.2010.5540195]
[17]Kim B,Yoo H,Sohn K.Exact order based feature descriptor for illumination robust image matching[J].Pattern Recognition,2013,46(12):3268-3278.[DOI:10.1016/j.patcog.2013.04.015]
[18]Fan B,Wu F C,Hu Z Y.Rotationally invariant descriptors using intensity order pooling[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(10):2031-2045.[DOI:10.1109/TPAMI.2011.277]
[19]Wang Z H,Fan B,Wu F C.Local intensity order pattern for feature description[C]//Proceedings of 2011 IEEE International Conference on Computer Vision.Barcelona,Spain:IEEE,2011:603-610.[DOI:10.1109/ICCV.2011.6126294]
[20]Mikolajczyk K,Tuytelaars T,Schmid C,et al.A comparison of affine region detectors[J].International Journal of Computer Vision,2005,65(1-2):43-72.[DOI:10.1007/s11263-005-3848-x]
[21]Mikolajczyk K,Schmid C.A performance evaluation of local descriptors[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2005,27(10):1615-1630.[DOI:10.1109/TPAMI.2005.188]