基于内容的图像检索关键技术的研究与实现
|
摘要
随着多媒体技术的发展和数字化应用的不断推广,基于内容的图像检索系统日益成为多媒体检索领域的研究热点。它具有良好的市场前景和研究潜力,通过对这些关键技术的研究,可以有效的提高图像检索的效率。
本文首先介绍了基于内容的图像检索研究的背景和意义,阐述了国内外发展现状进行分析,论述并分析了基于内容的图像检索相关关键技术,包括图像特征描述和图像局部特征。在图像特征描述中,阐述了图像的颜色、纹理和形状等低层特征。在图像局部特征中,阐述了SIFT算法,该算法以其出色的描述能力和鲁棒性,得到了大量应用。针对图像低层特征和高层语义之间的语义鸿沟问题,本文采用了“视觉词”的概念,通过对提取的SIFT特征进行聚类,得到图像视觉词,进而生成视觉词典。在聚类过程中,本文采用了聚类效果更好的FCM聚类算法,而不是常用的K均值聚类算法,不过由于计算复杂度更高,因此牺牲了一定的计算速度。同时,考虑到在聚类过程中丢失了图像空间信息,本文采用图像金字塔的方式加入空间信息,最后得到了图像特征向量：空间视觉词分布密度直方图。在进行图像的相似性度量时,本文采用了直方图相交法。
最后,本文将所采用的“SIFT+FCM+金字塔”方法在图像库中进行了测试,证明了本算法的有效性,并对仿真结果进行了分析,指明了下一步需要改进之处以及今后的研究方向。
With the development of multimedia technology and the digital application, content-based image retrieval system has become the focus of multimedia retrieval research. It has bright market prospective and research potential, the efficiency of image retrieval will be improved significantly through the research of those key technologies.
First, the background and meaning of content-based image retrieval is introduced in this thesis,which states the current image retrieval systems. Then it analysis the development status at home and abroad,through the analysis of those retrieval systems, it discusses the relative key technologies of content-based image retrieval.
In view of the theoretical principle, this thesis depicts several key technologies involved in image retrieval in detail including the image feature description, such as color, texture and shape feature, and the image local features. In image local features, SIFT algorithm possesses the remarkable descriptive power and robustness, and gains generous applications. In order to overcome the semantic gap between low level feature and high level semantic, this thesis introduces the concept "visual words", after abstracting the SIFT feature, and it uses the clustering algorithm to generate image visual words and processes to the next step—“visual dictionary”. In the clustering procedure, this thesis uses the FCM algorithm and the common used K-means clustering algorithm, which gains more accurate results. As the FCM algorithm has more computation complexity, it's calculation speed become lower. Meanwhile, in the clustering process, some image space information lost,and this thesis adopts the image pyramids algorithms to add space information, and finally get the image feature vectorspace visual word distribution density histogram. In the similarity measurement stage, this thesis uses the Histogram intersection method.
In the last part, this paper tests the “SIFT+FCM+PYRAMID” algorithms on the image library, and proves effectiveness, then it analysis the simulation result and point out the future research direction.
本文首先介绍了基于内容的图像检索研究的背景和意义,阐述了国内外发展现状进行分析,论述并分析了基于内容的图像检索相关关键技术,包括图像特征描述和图像局部特征。在图像特征描述中,阐述了图像的颜色、纹理和形状等低层特征。在图像局部特征中,阐述了SIFT算法,该算法以其出色的描述能力和鲁棒性,得到了大量应用。针对图像低层特征和高层语义之间的语义鸿沟问题,本文采用了“视觉词”的概念,通过对提取的SIFT特征进行聚类,得到图像视觉词,进而生成视觉词典。在聚类过程中,本文采用了聚类效果更好的FCM聚类算法,而不是常用的K均值聚类算法,不过由于计算复杂度更高,因此牺牲了一定的计算速度。同时,考虑到在聚类过程中丢失了图像空间信息,本文采用图像金字塔的方式加入空间信息,最后得到了图像特征向量：空间视觉词分布密度直方图。在进行图像的相似性度量时,本文采用了直方图相交法。
最后,本文将所采用的“SIFT+FCM+金字塔”方法在图像库中进行了测试,证明了本算法的有效性,并对仿真结果进行了分析,指明了下一步需要改进之处以及今后的研究方向。
With the development of multimedia technology and the digital application, content-based image retrieval system has become the focus of multimedia retrieval research. It has bright market prospective and research potential, the efficiency of image retrieval will be improved significantly through the research of those key technologies.
First, the background and meaning of content-based image retrieval is introduced in this thesis,which states the current image retrieval systems. Then it analysis the development status at home and abroad,through the analysis of those retrieval systems, it discusses the relative key technologies of content-based image retrieval.
In view of the theoretical principle, this thesis depicts several key technologies involved in image retrieval in detail including the image feature description, such as color, texture and shape feature, and the image local features. In image local features, SIFT algorithm possesses the remarkable descriptive power and robustness, and gains generous applications. In order to overcome the semantic gap between low level feature and high level semantic, this thesis introduces the concept "visual words", after abstracting the SIFT feature, and it uses the clustering algorithm to generate image visual words and processes to the next step—“visual dictionary”. In the clustering procedure, this thesis uses the FCM algorithm and the common used K-means clustering algorithm, which gains more accurate results. As the FCM algorithm has more computation complexity, it's calculation speed become lower. Meanwhile, in the clustering process, some image space information lost,and this thesis adopts the image pyramids algorithms to add space information, and finally get the image feature vectorspace visual word distribution density histogram. In the similarity measurement stage, this thesis uses the Histogram intersection method.
In the last part, this paper tests the “SIFT+FCM+PYRAMID” algorithms on the image library, and proves effectiveness, then it analysis the simulation result and point out the future research direction.
引文
[1]王惠锋,孙正兴,王箭.语义图像检索研究进展[J].计算机研究与发展,2002,Vol.39 No.5：513-523.
[2]向友军,谢胜利.图像检索技术综述[J].重庆邮电学院学报(自然科学版),2006,Vol.18 No.3：1-7.
[3]Kato T. Database architecture for content-based image retrieval[C]. Proc. of SPIE, San Jose, CA,USA,1992,112-123.
[4]黄祥林,沈兰荪.基于内容的图像检索技术研究[J].电子学报， 2002, 30(7)：1065-1071.
[5]Jain.P,KulisB.,Grauman,K. Fast image research for learned metrics,CVPR 2008,pp 1-8.
[6]In Kyu Park, Dong Yun,Sang UK Lee. Color image retrieval using hybrid graph representation[J]. Image and Vision Computing 2004,(17):465-474.
[7]Payne J S, Hepple while L, Stonham, T J. Perceptually based metrics for the evalution of textual image retrieval methods [A]. In Proceedings of IEEE International Conference on Multimedia Computing and System 99[C], Florence, Italy:IEEE Computer Society,2003:793-797.
[8]FLICKNER M, NIBLACK W, et al, Query by image and video content:the QBIC system[J].IEEE Computer,1995,28(9),23-32.
[9]John R SMITH, Shih-fu CHANG, VisualSEEK:a fully automated content-based image query system, ACM Multimedia 96, November 20,1996.
[10]Bach J Fuller C,GuPtaA, et al. The Virage Image SeachEngine:An Open Framework for Image Management[C]. In Proc. SPIE, Storage and Retrieval for Still Image and Video DatabaselV, vol.2670. SanJose, CA, USA,1996:76-87.
[11]PentlandA,Pieard R W, Sclarorr S. Photobook:Tools for content-based manipulation of image databases[J]. Storage and Retrieval for Image and Video Database II,1996:34-47.
[12]Mehrotra S, Rui Y, Ortega Metal.Supporting Content-based Queries over Image in MARS[C].In:Proc of IEEE,1997,632-633
[13]中国科学院成果鉴定报告.基于特征的多媒体信息检索系统MIREs(国家863高技术计划通信技术主题资助项目)1999,12,15.
[14]Wandell B A.The synthesis and analysis of color image.IEEE Transaction on Pattern analysis and machine intelligence.1987,9(1):2-13.
[15]John R. Smith.Shih-Fu Chang. Automated binary texture feature sets for image retrieval[C]. In Proc. IEEE Int. Conf. Acoust., Speech, and Signal Proc.1996.5,
[16]Haralick R M, Shanmugam K. Texture features for image classification.Trans. On system, manand cybernetics.1973, SMC-3(6):610-621.
[17]章毓晋,基于内容的视觉信息检索[M].北京：科学出版社,2003.
[18]Rui, Y., A. C.She, et al. Modified Fourier descriptors for shape representation-A Practicalapproach.Proceedings of 1 st International Workshop on Image Databases and Multimedia Search,1996. Amsterdam Netherlands:456-461.
[19]Harris C and Stephens M. A combined corner and edge deteetor. Proceedings of the Alvey Vision Conference. January 1998:147-151.
[20]D.Lowe.Distinctive Image features from Scale-invariant Key-Points.IJCV,60(2): 91-110,2004.
[21]D.Lowe.Object Recognition from Loeal Scale-invariant features. In Proc International Conference on Computer vision,Pages 1150-1157,1999.
[22]H.Bay. T.Tuytelaars. L.vanGool. Surf:Speed ed up robust features.Proeeedings of the 9th European conference on computer vision,2006,404-417.
[23]Zhu X. Q., Zhang H. J., New query refinement and semantics integrated image retrieval system with semiautomatic annotation scheme. Journal of Electronic Imaging,2001,16(3):533-566.
[24]Han J. W., Guo L., A new image retrieval model supporting query by semantics and example. IEEE Conference on Image Processing, "New York, USA, September,2002:31-38.
[25]Lu Y, Hu C, Zhou X, et al. A unified framework for semantics and feature based relevance feedback in image retrieval systems. Proc of ACM Multimedia conference,2000:31-37.
[26]Castleman K. R.,著,朱志刚等译,数字图像处理,电子工业出版社,北京,1998，261-304.
[27]章毓晋,图像工程上册-图像处理与分析,北京：清华大学出版社,1999.
[28]徐旭,朱淼良,梁倩卉,WaheedSajjad,一种用于CBIR系统的主色提取及表示方法,计算机辅助设计与图形学学报,1999,11(5)：385-388.
[29]孙兴华,基于内容的图像检索研究,博士论文,南京理工大学,2001.
[30]Swain M. J., Ballard D. H., Color indexing. Intl. J. on Computer Vision,1991, 7(1):11-32.
[31]Stricker M., Orengo M., Similarity of color images. In:Proceedings of SPIE Storage and Retrieval for Image and Video Database,1995,2420:381-392.
[32]John R. Smith.Shih-Fu Chang. Tools and techniques for color image retrieval. In Proc. OfSPIE:Storage and Retrieval for Image and Video Database.1995.Vol.2670.
[33]G. Pass. R. Zabih. Histogram refinement for content-based image retrieval[C]. IEEE Workshop on Applications of Computer Vision.1996:96-102.
[34]Robert M. Haralick. K. Shanmugam.Its'hakDinstein. Texture features for image classification. IEEE Trans. On Sys, Man, and Cyb, SMC-3(6):610-621,1973.
[35]H. Tamura, S. Mori, and T. Yamawaki, Texture features corresponding to visual perception. IEEE Trans. On Systems, Man, and Cybernetics.1978.6. Vol. Smc-8 No.6.
[36]Lain A., Fan J. Texture classification by wavelet packet signatures. IEEE Trans. on PAMI,1993,15:1186-1191.
[37]Chang T, Kuo C. C. J., Texture analysis and classification with tree-structured wavelet transform. IEEE Trans, on Image Processing,1993,2:429-441.
[38]DengshengZhang. Review of shape representation and description techniques[J]. Pattern Recognition 37 (2009):1-19.
[39]D.S. Guru,P. Nagabhushan. Symbolic representation of two-dimensional shape[J]. Pattern Recognition Letters.28(2009):144-155.
[40]M.K. Hu. Visual pattern recognition by moment invariants[M],IRE Trans. Inf. Theory IT-8 (1962):179-187.
[41]杨利峰,胡茂林.基于非监督纹理分割下的图像查询,计算机技术与发展,2006,16(5).
[42]J.winn, A.Criminisi. and T.Minka.Object categorization by learned universal visual dietionary. In International Conference on Computer vision 2005.
[43]F.Moosmann. B.Triggs, and F.Jurie.Randomized clustering forests for building fast and discriminative visual vocabularies. In NIPS.2010.
[44]K.Mikolajczyk, C.Schmid.Indexing Based on Scale In variant Interest points. Proceedings of the 8th International Conference on Cpmputer Vision,ancouver,Canda， 2001,525-531.
[45]K.Mikolajczyk,C.Schmid.Scale& Affine invariant interest point detectors. Intemational Journal of Computer Vision,2004,60(1):63-86.
[46]D.Marr, E.Hildreth.Theory of edge deteetion.Proeeedings of the Royal Society of London (SeriesB), Biological Scienees,1980,207(1167):187-217.
[47]向友军,谢胜利.图像检索技术综述[J].重庆邮电学院学报(自然科学版), 2006,Vol.18 No.3：1-7.
[8]吴林,郭大勇,施克仁等.改进的FCM在人脑MR图像分割中的应用.清华大学学报(自然科学版),2004, 44(2)：157-159.
[49]O.Boiman,E.Shechtman,andM.Irani.In defenseofnearest-neighborbasedimage classification.In CVPR,2010.2,5,7.
[50]吴林,郭大勇,施克仁等.改进的FCM在人脑MR图像分割中的应用.清华大学学报(自然科学版),2004, 44(2)：157-159.
[51]Zadeh,1.A.,Fuzzy sets.Information and Control,1965.8:338-353.
[52]孙增,智能控制理论与技术,北京,清华大学出版社,1997, P16.
[53]DunnJ.C.A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well Separated Clustert [J].J Cybernet,1974(3):32-57.
[54]Bezdek J C. A Convergence Theorem for The Fuzzy ISODATA Clustering Algorithm [J]. IEEE PAMI,1980,1(2):1-8.
[55]沈清,汤霖,模式识别导论,长沙,国防科技大学出版,1991, P106.
[56]于剑,程乾生,关于FCM算法中的权重指数m的一点注记,电子学报,2008,Vol.31,No.3,P478-480.
[57]宫改云,高新波,伍忠东,聚类算法中模糊加权指数m的优选方法,模糊系统与数学,2005,Vol.19, No.1,P143-1480.
[58]于剑,论模糊C均值法的模糊指标,计算机学报,2003,Vol.26,No.8,P968-973.
[59]W H.Hsu and S-F Chang.Visual Cue Cluster Construction via Information Bottleneck Principle and Kernel Density Estimation.In Proc.of CIVR,2009.
[60]L.Fei-Fei and P.Perona.A Bayesian hierarchical model for learning natural scene categories. In Proc.VPR,2005.
[61]A.Oliva and A.Torralba. Modeling the shape of the scene:a holistic representation of the spatial envelope. IJCV,42(3):145-175,2001.
[2]向友军,谢胜利.图像检索技术综述[J].重庆邮电学院学报(自然科学版),2006,Vol.18 No.3：1-7.
[3]Kato T. Database architecture for content-based image retrieval[C]. Proc. of SPIE, San Jose, CA,USA,1992,112-123.
[4]黄祥林,沈兰荪.基于内容的图像检索技术研究[J].电子学报， 2002, 30(7)：1065-1071.
[5]Jain.P,KulisB.,Grauman,K. Fast image research for learned metrics,CVPR 2008,pp 1-8.
[6]In Kyu Park, Dong Yun,Sang UK Lee. Color image retrieval using hybrid graph representation[J]. Image and Vision Computing 2004,(17):465-474.
[7]Payne J S, Hepple while L, Stonham, T J. Perceptually based metrics for the evalution of textual image retrieval methods [A]. In Proceedings of IEEE International Conference on Multimedia Computing and System 99[C], Florence, Italy:IEEE Computer Society,2003:793-797.
[8]FLICKNER M, NIBLACK W, et al, Query by image and video content:the QBIC system[J].IEEE Computer,1995,28(9),23-32.
[9]John R SMITH, Shih-fu CHANG, VisualSEEK:a fully automated content-based image query system, ACM Multimedia 96, November 20,1996.
[10]Bach J Fuller C,GuPtaA, et al. The Virage Image SeachEngine:An Open Framework for Image Management[C]. In Proc. SPIE, Storage and Retrieval for Still Image and Video DatabaselV, vol.2670. SanJose, CA, USA,1996:76-87.
[11]PentlandA,Pieard R W, Sclarorr S. Photobook:Tools for content-based manipulation of image databases[J]. Storage and Retrieval for Image and Video Database II,1996:34-47.
[12]Mehrotra S, Rui Y, Ortega Metal.Supporting Content-based Queries over Image in MARS[C].In:Proc of IEEE,1997,632-633
[13]中国科学院成果鉴定报告.基于特征的多媒体信息检索系统MIREs(国家863高技术计划通信技术主题资助项目)1999,12,15.
[14]Wandell B A.The synthesis and analysis of color image.IEEE Transaction on Pattern analysis and machine intelligence.1987,9(1):2-13.
[15]John R. Smith.Shih-Fu Chang. Automated binary texture feature sets for image retrieval[C]. In Proc. IEEE Int. Conf. Acoust., Speech, and Signal Proc.1996.5,
[16]Haralick R M, Shanmugam K. Texture features for image classification.Trans. On system, manand cybernetics.1973, SMC-3(6):610-621.
[17]章毓晋,基于内容的视觉信息检索[M].北京：科学出版社,2003.
[18]Rui, Y., A. C.She, et al. Modified Fourier descriptors for shape representation-A Practicalapproach.Proceedings of 1 st International Workshop on Image Databases and Multimedia Search,1996. Amsterdam Netherlands:456-461.
[19]Harris C and Stephens M. A combined corner and edge deteetor. Proceedings of the Alvey Vision Conference. January 1998:147-151.
[20]D.Lowe.Distinctive Image features from Scale-invariant Key-Points.IJCV,60(2): 91-110,2004.
[21]D.Lowe.Object Recognition from Loeal Scale-invariant features. In Proc International Conference on Computer vision,Pages 1150-1157,1999.
[22]H.Bay. T.Tuytelaars. L.vanGool. Surf:Speed ed up robust features.Proeeedings of the 9th European conference on computer vision,2006,404-417.
[23]Zhu X. Q., Zhang H. J., New query refinement and semantics integrated image retrieval system with semiautomatic annotation scheme. Journal of Electronic Imaging,2001,16(3):533-566.
[24]Han J. W., Guo L., A new image retrieval model supporting query by semantics and example. IEEE Conference on Image Processing, "New York, USA, September,2002:31-38.
[25]Lu Y, Hu C, Zhou X, et al. A unified framework for semantics and feature based relevance feedback in image retrieval systems. Proc of ACM Multimedia conference,2000:31-37.
[26]Castleman K. R.,著,朱志刚等译,数字图像处理,电子工业出版社,北京,1998，261-304.
[27]章毓晋,图像工程上册-图像处理与分析,北京：清华大学出版社,1999.
[28]徐旭,朱淼良,梁倩卉,WaheedSajjad,一种用于CBIR系统的主色提取及表示方法,计算机辅助设计与图形学学报,1999,11(5)：385-388.
[29]孙兴华,基于内容的图像检索研究,博士论文,南京理工大学,2001.
[30]Swain M. J., Ballard D. H., Color indexing. Intl. J. on Computer Vision,1991, 7(1):11-32.
[31]Stricker M., Orengo M., Similarity of color images. In:Proceedings of SPIE Storage and Retrieval for Image and Video Database,1995,2420:381-392.
[32]John R. Smith.Shih-Fu Chang. Tools and techniques for color image retrieval. In Proc. OfSPIE:Storage and Retrieval for Image and Video Database.1995.Vol.2670.
[33]G. Pass. R. Zabih. Histogram refinement for content-based image retrieval[C]. IEEE Workshop on Applications of Computer Vision.1996:96-102.
[34]Robert M. Haralick. K. Shanmugam.Its'hakDinstein. Texture features for image classification. IEEE Trans. On Sys, Man, and Cyb, SMC-3(6):610-621,1973.
[35]H. Tamura, S. Mori, and T. Yamawaki, Texture features corresponding to visual perception. IEEE Trans. On Systems, Man, and Cybernetics.1978.6. Vol. Smc-8 No.6.
[36]Lain A., Fan J. Texture classification by wavelet packet signatures. IEEE Trans. on PAMI,1993,15:1186-1191.
[37]Chang T, Kuo C. C. J., Texture analysis and classification with tree-structured wavelet transform. IEEE Trans, on Image Processing,1993,2:429-441.
[38]DengshengZhang. Review of shape representation and description techniques[J]. Pattern Recognition 37 (2009):1-19.
[39]D.S. Guru,P. Nagabhushan. Symbolic representation of two-dimensional shape[J]. Pattern Recognition Letters.28(2009):144-155.
[40]M.K. Hu. Visual pattern recognition by moment invariants[M],IRE Trans. Inf. Theory IT-8 (1962):179-187.
[41]杨利峰,胡茂林.基于非监督纹理分割下的图像查询,计算机技术与发展,2006,16(5).
[42]J.winn, A.Criminisi. and T.Minka.Object categorization by learned universal visual dietionary. In International Conference on Computer vision 2005.
[43]F.Moosmann. B.Triggs, and F.Jurie.Randomized clustering forests for building fast and discriminative visual vocabularies. In NIPS.2010.
[44]K.Mikolajczyk, C.Schmid.Indexing Based on Scale In variant Interest points. Proceedings of the 8th International Conference on Cpmputer Vision,ancouver,Canda， 2001,525-531.
[45]K.Mikolajczyk,C.Schmid.Scale& Affine invariant interest point detectors. Intemational Journal of Computer Vision,2004,60(1):63-86.
[46]D.Marr, E.Hildreth.Theory of edge deteetion.Proeeedings of the Royal Society of London (SeriesB), Biological Scienees,1980,207(1167):187-217.
[47]向友军,谢胜利.图像检索技术综述[J].重庆邮电学院学报(自然科学版), 2006,Vol.18 No.3：1-7.
[8]吴林,郭大勇,施克仁等.改进的FCM在人脑MR图像分割中的应用.清华大学学报(自然科学版),2004, 44(2)：157-159.
[49]O.Boiman,E.Shechtman,andM.Irani.In defenseofnearest-neighborbasedimage classification.In CVPR,2010.2,5,7.
[50]吴林,郭大勇,施克仁等.改进的FCM在人脑MR图像分割中的应用.清华大学学报(自然科学版),2004, 44(2)：157-159.
[51]Zadeh,1.A.,Fuzzy sets.Information and Control,1965.8:338-353.
[52]孙增,智能控制理论与技术,北京,清华大学出版社,1997, P16.
[53]DunnJ.C.A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well Separated Clustert [J].J Cybernet,1974(3):32-57.
[54]Bezdek J C. A Convergence Theorem for The Fuzzy ISODATA Clustering Algorithm [J]. IEEE PAMI,1980,1(2):1-8.
[55]沈清,汤霖,模式识别导论,长沙,国防科技大学出版,1991, P106.
[56]于剑,程乾生,关于FCM算法中的权重指数m的一点注记,电子学报,2008,Vol.31,No.3,P478-480.
[57]宫改云,高新波,伍忠东,聚类算法中模糊加权指数m的优选方法,模糊系统与数学,2005,Vol.19, No.1,P143-1480.
[58]于剑,论模糊C均值法的模糊指标,计算机学报,2003,Vol.26,No.8,P968-973.
[59]W H.Hsu and S-F Chang.Visual Cue Cluster Construction via Information Bottleneck Principle and Kernel Density Estimation.In Proc.of CIVR,2009.
[60]L.Fei-Fei and P.Perona.A Bayesian hierarchical model for learning natural scene categories. In Proc.VPR,2005.
[61]A.Oliva and A.Torralba. Modeling the shape of the scene:a holistic representation of the spatial envelope. IJCV,42(3):145-175,2001.