结合底层特征和高层语义的图像检索技术研究
|
摘要
随着计算机技术和多媒体技术的快速发展,多媒体图像的数量也以得到了极大地增长,如何从海量的图像库中快速、准确的检索到所需求的图像成为了当今多媒体技术中研究的热点问题。传统的基于文本的图像检索技术需要管理员手工对图像进行标注,不仅消耗了大量的人力,而且人工标注图像的主观性很大,对于不同的管理员,标注的结果可能不同。基于内容的图像检索技术是依靠图像的低层视觉特征(颜色、纹理、形状等)来进行检索的,但是人对图像的认识是一个利用自己的先验知识推理图像语义的过程,这样导致了图像的底层视觉特征和图像语义之间的“语义鸿沟”。
为了减小“语义鸿沟”,本文将图像的高层语义和底层视觉特征结合起来,利用支持向量机(SVM)将图像的底层特征映射为高层语义。
本文首先对语义的层次模型进行了分析,并且介绍了提取图像语义的一些常用方法。在分析了图像颜色、纹理、形状等特征提取方法的基础上,提出采用结合图像边缘和角点信息的低层特征提取方法,分别用不变矩和环形颜色直方图来表示图像的边缘和角点信息。
本文重点研究了支持向量机的多分类技术,针对一些传统方法支持向量机多分类的缺点,例如:正负样本分布不均匀、识别率低、训练时间长等,提出了一种新的二叉树结构的SVM分类方法。以样本的空间分布为切入点,利用K-Mean聚类分析样本语义类之间的空间分布,采用聚类中心的欧氏距离作为量度,在树形结构SVM的根节点中首先确定空间距离最大的两个类别,将这两个类别分别确定为SVM正类和负类的中心,其他类根据它们与此两类的距离被分配到其对应的SVM类别中。对其他结点SVM类别,再按照根节点同样方式进行分类,直到最后得到单一的类别。以这种分配SVM正负类别的方式训练树形SVM,正负类别比较均匀,先分离开距离较远的类别,避免了它们对后续分类的干扰,提高了分类的准确率,而且除了根节点之外的节点中SVM所有数据量比其他树形结构方法都有很大减少,缩短了SVM的训练时间。实验结果表明,该方法在保证准确率的同时可以在较大程度上缩短图像检索时间。
With the rapid development of computer and multimedia technology, the number of multimedia image mushrooms. How to retrieval the image you want quickly and accurately in a gigantic image database is a crucial problem of the multimedia technology research. Traditional keywords-based image retrieval technology need manager annotate images by hand, so it not only cost so much human labors, but also have subjectivity, different manager maybe have different label for the same image. The content-based image retrieval technology mainly searches image by visual contents (color, texture, shape and so no). But people understands a image is a process that he uses his knowledge to speculate semantics of the image, thus, it lead a“semantics gap”between low level features and image semantics.
To reduce“semantics gap”, a method that combines the high level semantic and low level feature is proposed. It utilizes support vector machine (SVM) to transform low level features that are extracted from an image into high level semantics.
In this paper, the semantic hierarchy of an image is analyzed and some classical methods of image semantic extracting are presented. On the basis of analysis some methods of extracting color feature, texture feature and shape feature propose, a low level feature extracting method that combines image edges and corners is proposed. It uses moment invariants express image edge and ring-shaped color histogram to express information of corners.
This paper focuses on the multi-classification of SVM. In order to overcome the faults of traditional multi-classification of SVM, such as positive sample and negative sample are not balanceable, low recognition rate, train time so long, etc. a new tree structure SVM is proposed. Based on space distribution of the sample images, K-mean clustering is used to analyze space distribution among sample images semantic classification, and Euclidean distances among each clustering center are used as a tool to separate the classes. Firstly, two classes are classified, which have biggest distance of positive sample and negative sample of SVM in the root node of tree structure SVM. Then, the other classes will be classified to the corresponding SVM node if their distances are shorter to the one of two classes. For the other nodes, the classes are classified to two classes again. This step is repeated until only one class in the node. This distribution of positive sample and negative sample of SVM keep the balance of positive sample and negative sample. It classifies the two classes that have biggest distance to avoid disturbing other classification and increases the accuracy of classification. Moreover, it decreases the number of nodes of SVM and the training time of SVM. The experimental results show that the proposed method not only can improve image retrieval accuracy, but also reduce retrieval time.
为了减小“语义鸿沟”,本文将图像的高层语义和底层视觉特征结合起来,利用支持向量机(SVM)将图像的底层特征映射为高层语义。
本文首先对语义的层次模型进行了分析,并且介绍了提取图像语义的一些常用方法。在分析了图像颜色、纹理、形状等特征提取方法的基础上,提出采用结合图像边缘和角点信息的低层特征提取方法,分别用不变矩和环形颜色直方图来表示图像的边缘和角点信息。
本文重点研究了支持向量机的多分类技术,针对一些传统方法支持向量机多分类的缺点,例如:正负样本分布不均匀、识别率低、训练时间长等,提出了一种新的二叉树结构的SVM分类方法。以样本的空间分布为切入点,利用K-Mean聚类分析样本语义类之间的空间分布,采用聚类中心的欧氏距离作为量度,在树形结构SVM的根节点中首先确定空间距离最大的两个类别,将这两个类别分别确定为SVM正类和负类的中心,其他类根据它们与此两类的距离被分配到其对应的SVM类别中。对其他结点SVM类别,再按照根节点同样方式进行分类,直到最后得到单一的类别。以这种分配SVM正负类别的方式训练树形SVM,正负类别比较均匀,先分离开距离较远的类别,避免了它们对后续分类的干扰,提高了分类的准确率,而且除了根节点之外的节点中SVM所有数据量比其他树形结构方法都有很大减少,缩短了SVM的训练时间。实验结果表明,该方法在保证准确率的同时可以在较大程度上缩短图像检索时间。
With the rapid development of computer and multimedia technology, the number of multimedia image mushrooms. How to retrieval the image you want quickly and accurately in a gigantic image database is a crucial problem of the multimedia technology research. Traditional keywords-based image retrieval technology need manager annotate images by hand, so it not only cost so much human labors, but also have subjectivity, different manager maybe have different label for the same image. The content-based image retrieval technology mainly searches image by visual contents (color, texture, shape and so no). But people understands a image is a process that he uses his knowledge to speculate semantics of the image, thus, it lead a“semantics gap”between low level features and image semantics.
To reduce“semantics gap”, a method that combines the high level semantic and low level feature is proposed. It utilizes support vector machine (SVM) to transform low level features that are extracted from an image into high level semantics.
In this paper, the semantic hierarchy of an image is analyzed and some classical methods of image semantic extracting are presented. On the basis of analysis some methods of extracting color feature, texture feature and shape feature propose, a low level feature extracting method that combines image edges and corners is proposed. It uses moment invariants express image edge and ring-shaped color histogram to express information of corners.
This paper focuses on the multi-classification of SVM. In order to overcome the faults of traditional multi-classification of SVM, such as positive sample and negative sample are not balanceable, low recognition rate, train time so long, etc. a new tree structure SVM is proposed. Based on space distribution of the sample images, K-mean clustering is used to analyze space distribution among sample images semantic classification, and Euclidean distances among each clustering center are used as a tool to separate the classes. Firstly, two classes are classified, which have biggest distance of positive sample and negative sample of SVM in the root node of tree structure SVM. Then, the other classes will be classified to the corresponding SVM node if their distances are shorter to the one of two classes. For the other nodes, the classes are classified to two classes again. This step is repeated until only one class in the node. This distribution of positive sample and negative sample of SVM keep the balance of positive sample and negative sample. It classifies the two classes that have biggest distance to avoid disturbing other classification and increases the accuracy of classification. Moreover, it decreases the number of nodes of SVM and the training time of SVM. The experimental results show that the proposed method not only can improve image retrieval accuracy, but also reduce retrieval time.
引文
[1] Y. Alp. Aslandogan, Clement. T. Yu. Techniques and systems for image and video retrieval[J]. IEEE Transaction on Knowledge and Data Engineering,1999, 11(1): 56-63.
[2] W. M. Amoid, W. Marce, S. Simone. Content-based image retrieval at the end of the early years[J]. IEEE Transaction on Pattern Analysis and Machine Intelligence,2000, 22(12): 1349-1379.
[3] Veltkamp R C, Tanase Mierla. Content-Based Image Retrieval Systems[J]. A Survey Technical Report UU-CS-2000-34.2000.10.
[4] M.Flickner, H.Sawhney, W. Niblack, et al. Query by Image and Video Content: The QBIC System[J]. Computer, 1995, 28(9): 23-32.
[5] A. Gupta and R. Jain. Visual Information Retrieval[J].Comm.ACM.1997,40(5):70~79.
[6] Yong Rui, S. Thomas Huang and Sharad Mehrotra. Content-based Image Retrieval With Relevance Feedback in Mars[J].IEEE.1997(2):815– 818.
[7] C. F. Herot. Spatial Management of Data[J]. ACM Trans.1980,(5):493-514.
[8] R. John Smith and Shih-Fu Chang. VisualSEEK: A Fully Automated C based Image Query System. ACM Multimedia[M],Boston,MA.1996.
[9] R. John Smith and Shih-Fu Chang. Visually Searching the Web for Content[J]. IEEE Multimedia magazine.1997.4(3):12-20.
[10] W.Y.Ma and B.S.Manjunath. Netra: A toolbox for Navigating Large Image Databases[C]. IEEE International Conference on Image Processing.1997, 1: 568-671.
[11]周璐,基于高级语义特征的图像检索技术研究.辽宁:辽宁师范大学,2008.
[12]付岩,王耀威,王伟强,高文.SVM用于基于内容的自然图像分类和检索[J],计算机学报. 2003, 26(10): 1261~1265.
[13] J P Eakins. Automatic image content retrieval—Are we getting anywhere?[J] In: Proc of 3rd Int’l Conf on Electronic Library and Visual Information Research. De Montfort University, Milton Keynes: A slib, 1996: 123~135.
[14] V N Gudivada, V V Raghavan. Content-based image retrieval system[J]. Computer, 1995, 28(9): 18~20.
[15] T. Hermes, et al. Image retrieval for information systems[J]. In Storage and Retrieval for Image and Video Databases III, Proc. SPIE 2420.1995:394-405.
[16]王惠锋,金翔宇,孙正兴.语义图像检索研究进展[J].计算机研究与发展.2002, 39(5):513~522.
[17]沈玉利,任建峰,郭雷.一种由底层视觉特征获取高层语义的图像检索方法[J]计算机工程.2005.2 172-174.
[18] V.N. Vapnik著,张学工译,统计学习理论的本质[M].清华大学出版社.2000.85-116.
[19] M LIU. Overview of the ROL2 deductive object- oriented database system[C]. Proceedings of the 30th International Conference on Technology of Object-Oriented Languages & Systems, IEEE Computer Society Press, 1999: 63～72.
[20] Bimbo A. Visual Information Retrieval [M ]. San Francisco, CA,USA: Morgan Kaufmann Publishers, 1999.
[21] Han Jun Wei, Guo Lei. A new image retrieval system supporting query by semantics and example[J],IC IP01(2):953-956.
[22] Y.A.Aslandogan. Using Semantic Contents and WordNet in Image Retrieval[J], Proceeding of ACM SIGIR97,1999, 286-295.
[23] Hong Wu, Mingjing Li, HongJiang Zhang et al. Improving Image Retrieval with Semantic Classification Using Relevance Feedback[J].VDB 2002,327-339.
[24]章毓晋.基于内容的视觉信息检索[M].科学出版社,2003:7.
[25]章毓晋,图像工程(上册)-图像处理与分析[M],清华大学出版社,1999.
[26]夏良正,数字图像处理[M],东南大学出版社,1999.
[27]章毓晋,图像理解与计算机视觉[M],清华人学出版社,2000.8.
[28] Kenneth. R. Castleman,数字图像处理[M],电子工业出版社,1998.
[29]雷方元等,一种基于颜色块直方图的图像检索方法[J],计算机应用,2004.5:173-194
[30] Swain MJ, Ballard DH. Color indexing[C].International Journal of Computer Vision,1991,7(1):11-32.
[31] Hafner J, Sawhney H S, et al. Efficient color histogram indexing for quadratic form distance function[J].IEEE Trans on Pattern Analysis and Machine Intelligence,1995,17(7):729-736.
[32] M.A. Stricker, M.Orengo. Similarity of color images[J]. Proc of SPIE: Storage and retrieval for Images and Video databasesⅢ. San Jose.CA.1995.2420:381-392.
[33] Smith J R, Chang S-F. Tools and techniques for color image retrieval[J], SPIE Storageand Retrieval,1996:426-437.
[34]邰晓英,王李冬,巴特尔.基于小波纹理、语义特征和相关反馈的医学图像检索[J].电路与系统学报.2007.12(4):24-30.
[35] N.RMudigonda, R.M.Rangayyan, J.E.L.Desautels. Gradient and texture analysis for the classification of mammo graphic masses[J].IEEE Transactions on Medical Imaging.2000.19(10):1032-1043.
[36]吕娜,孙杨民,黄国丰.对图像检索应用概况的研究[J].情报科学.2002,20(3):1-8.
[37]徐建华.图像处理分析[M].北京:科学出版社,1992.
[38]邬浩,潘云鹤,庄越挺,杨宇艇.基于对象形状的图像查询技术[J].软件学报,1998,9(5):344-349
[39] D.S. Zbang, GJ. Lu. Shape-based Image Processing: Image Communication[J].Retrieval Using Generic Fourier Descriptor.2002,17:825-848.
[40]朱玉燕,尚振宏.角点检测技术研究与进展[J].电脑开发与应用,2010,23(13): 40-47.
[41] Harris C, Stephens M. A combined corner and edge detector[C]. Matthews M M. Proceedings of the Fourth Alvey Vision Conference. Manchester: the University of Sheffield Printing Unit, 1988: 147-151.
[42]马力.基于支持向量机的视频对话自动分类方法研究[D].长沙:国防科技技术大学硕士学位论文,2007.
[43]杜晶.基于支持向量机的车牌字符识别研究[D].邯郸:河北工程大学硕士学位论文,2008.
[44]肖慧玲.支持向量机在医学图像处理中的应用[D].西安:电子科技大学,2008.
[45]梁俊芝.关于支持向量机方法的几点研究[D].西安:西北大学,2009.
[46]谢怡飞.基于SVM语义分类和视觉特征提取的图像检索方法研究[D].哈尔滨:哈尔滨工业大学, 2007.
[47] Xiang Tian, Feiqi Deng. An Improved Multi-class SVM Algorithm and Its Application to the Credit Scoring Model[C].Proceedings of the 5th World Congress on Intelligent Control and Automation.2004:303~318.
[48] L. Bolton, C. Cortes, J. Denker, et al. Comparison of Classifier Methods: A Case Study in Handwriting Digit Recognition[C]. International Conference on Pattern Recognition, IEEE Computer Society Press.1994:77~87.
[49]耿苑,郭雷.结合低层特征和高层语义的图像检索系统[D].西安:西北工业大学硕士毕业论文,2004 .
[50]胡迎松,张海龙.一种基于语义网络的图像检索方法[J].微机处理, 2008, 27(3): 81-84.
[51] Ye Lu, Chunhui Hu, Xingquan Zhu, et al. A Unified Framework for Semantics and Feature Based Relevance Feedback in Image Retrieval Systems[C]. Proceedings of the eighth ACM international conference on Multimedia, Los Angeles, California, 2000:31-37.
[52] Xiaohang Ma,Dianhui Wang. Semantics Modeling Based Image Retrieval System Using Neural Networks[C], IEEE International Conference on Image Processing, 2005: I-1165-8.
[53] Xue Gao, Jian yu Chen. A Semantic Modeling Approach for Medical Image Semantic Retrieval Using Hybrid Bayesian Networks[C], Proceedings of the Sixth International Conference on Intelligent Systems Design and Applications, 2006: 482-487.
[54] Guixiong Liu, Xiongping Zhang. Multi-Class Classification of Support Vector Machines Based on Double Binary Tree[C], Fourth International Conference on Natural Computation, 2008:103-105.
[55] B. Fei and J. Liu, Binary tree of SVM: A new fast Multi-class Training and Classification Algorithm[C], IEEE Transactions on Neural Networks, vol.17, no.3, pp.696 -704, May 2006.
[56] J. Platt. Fast training of support vector machines using sequential minimal optimization[J], In Advances in Kernel Methods - Support Vector Learning, 1999:185-208.
[2] W. M. Amoid, W. Marce, S. Simone. Content-based image retrieval at the end of the early years[J]. IEEE Transaction on Pattern Analysis and Machine Intelligence,2000, 22(12): 1349-1379.
[3] Veltkamp R C, Tanase Mierla. Content-Based Image Retrieval Systems[J]. A Survey Technical Report UU-CS-2000-34.2000.10.
[4] M.Flickner, H.Sawhney, W. Niblack, et al. Query by Image and Video Content: The QBIC System[J]. Computer, 1995, 28(9): 23-32.
[5] A. Gupta and R. Jain. Visual Information Retrieval[J].Comm.ACM.1997,40(5):70~79.
[6] Yong Rui, S. Thomas Huang and Sharad Mehrotra. Content-based Image Retrieval With Relevance Feedback in Mars[J].IEEE.1997(2):815– 818.
[7] C. F. Herot. Spatial Management of Data[J]. ACM Trans.1980,(5):493-514.
[8] R. John Smith and Shih-Fu Chang. VisualSEEK: A Fully Automated C based Image Query System. ACM Multimedia[M],Boston,MA.1996.
[9] R. John Smith and Shih-Fu Chang. Visually Searching the Web for Content[J]. IEEE Multimedia magazine.1997.4(3):12-20.
[10] W.Y.Ma and B.S.Manjunath. Netra: A toolbox for Navigating Large Image Databases[C]. IEEE International Conference on Image Processing.1997, 1: 568-671.
[11]周璐,基于高级语义特征的图像检索技术研究.辽宁:辽宁师范大学,2008.
[12]付岩,王耀威,王伟强,高文.SVM用于基于内容的自然图像分类和检索[J],计算机学报. 2003, 26(10): 1261~1265.
[13] J P Eakins. Automatic image content retrieval—Are we getting anywhere?[J] In: Proc of 3rd Int’l Conf on Electronic Library and Visual Information Research. De Montfort University, Milton Keynes: A slib, 1996: 123~135.
[14] V N Gudivada, V V Raghavan. Content-based image retrieval system[J]. Computer, 1995, 28(9): 18~20.
[15] T. Hermes, et al. Image retrieval for information systems[J]. In Storage and Retrieval for Image and Video Databases III, Proc. SPIE 2420.1995:394-405.
[16]王惠锋,金翔宇,孙正兴.语义图像检索研究进展[J].计算机研究与发展.2002, 39(5):513~522.
[17]沈玉利,任建峰,郭雷.一种由底层视觉特征获取高层语义的图像检索方法[J]计算机工程.2005.2 172-174.
[18] V.N. Vapnik著,张学工译,统计学习理论的本质[M].清华大学出版社.2000.85-116.
[19] M LIU. Overview of the ROL2 deductive object- oriented database system[C]. Proceedings of the 30th International Conference on Technology of Object-Oriented Languages & Systems, IEEE Computer Society Press, 1999: 63～72.
[20] Bimbo A. Visual Information Retrieval [M ]. San Francisco, CA,USA: Morgan Kaufmann Publishers, 1999.
[21] Han Jun Wei, Guo Lei. A new image retrieval system supporting query by semantics and example[J],IC IP01(2):953-956.
[22] Y.A.Aslandogan. Using Semantic Contents and WordNet in Image Retrieval[J], Proceeding of ACM SIGIR97,1999, 286-295.
[23] Hong Wu, Mingjing Li, HongJiang Zhang et al. Improving Image Retrieval with Semantic Classification Using Relevance Feedback[J].VDB 2002,327-339.
[24]章毓晋.基于内容的视觉信息检索[M].科学出版社,2003:7.
[25]章毓晋,图像工程(上册)-图像处理与分析[M],清华大学出版社,1999.
[26]夏良正,数字图像处理[M],东南大学出版社,1999.
[27]章毓晋,图像理解与计算机视觉[M],清华人学出版社,2000.8.
[28] Kenneth. R. Castleman,数字图像处理[M],电子工业出版社,1998.
[29]雷方元等,一种基于颜色块直方图的图像检索方法[J],计算机应用,2004.5:173-194
[30] Swain MJ, Ballard DH. Color indexing[C].International Journal of Computer Vision,1991,7(1):11-32.
[31] Hafner J, Sawhney H S, et al. Efficient color histogram indexing for quadratic form distance function[J].IEEE Trans on Pattern Analysis and Machine Intelligence,1995,17(7):729-736.
[32] M.A. Stricker, M.Orengo. Similarity of color images[J]. Proc of SPIE: Storage and retrieval for Images and Video databasesⅢ. San Jose.CA.1995.2420:381-392.
[33] Smith J R, Chang S-F. Tools and techniques for color image retrieval[J], SPIE Storageand Retrieval,1996:426-437.
[34]邰晓英,王李冬,巴特尔.基于小波纹理、语义特征和相关反馈的医学图像检索[J].电路与系统学报.2007.12(4):24-30.
[35] N.RMudigonda, R.M.Rangayyan, J.E.L.Desautels. Gradient and texture analysis for the classification of mammo graphic masses[J].IEEE Transactions on Medical Imaging.2000.19(10):1032-1043.
[36]吕娜,孙杨民,黄国丰.对图像检索应用概况的研究[J].情报科学.2002,20(3):1-8.
[37]徐建华.图像处理分析[M].北京:科学出版社,1992.
[38]邬浩,潘云鹤,庄越挺,杨宇艇.基于对象形状的图像查询技术[J].软件学报,1998,9(5):344-349
[39] D.S. Zbang, GJ. Lu. Shape-based Image Processing: Image Communication[J].Retrieval Using Generic Fourier Descriptor.2002,17:825-848.
[40]朱玉燕,尚振宏.角点检测技术研究与进展[J].电脑开发与应用,2010,23(13): 40-47.
[41] Harris C, Stephens M. A combined corner and edge detector[C]. Matthews M M. Proceedings of the Fourth Alvey Vision Conference. Manchester: the University of Sheffield Printing Unit, 1988: 147-151.
[42]马力.基于支持向量机的视频对话自动分类方法研究[D].长沙:国防科技技术大学硕士学位论文,2007.
[43]杜晶.基于支持向量机的车牌字符识别研究[D].邯郸:河北工程大学硕士学位论文,2008.
[44]肖慧玲.支持向量机在医学图像处理中的应用[D].西安:电子科技大学,2008.
[45]梁俊芝.关于支持向量机方法的几点研究[D].西安:西北大学,2009.
[46]谢怡飞.基于SVM语义分类和视觉特征提取的图像检索方法研究[D].哈尔滨:哈尔滨工业大学, 2007.
[47] Xiang Tian, Feiqi Deng. An Improved Multi-class SVM Algorithm and Its Application to the Credit Scoring Model[C].Proceedings of the 5th World Congress on Intelligent Control and Automation.2004:303~318.
[48] L. Bolton, C. Cortes, J. Denker, et al. Comparison of Classifier Methods: A Case Study in Handwriting Digit Recognition[C]. International Conference on Pattern Recognition, IEEE Computer Society Press.1994:77~87.
[49]耿苑,郭雷.结合低层特征和高层语义的图像检索系统[D].西安:西北工业大学硕士毕业论文,2004 .
[50]胡迎松,张海龙.一种基于语义网络的图像检索方法[J].微机处理, 2008, 27(3): 81-84.
[51] Ye Lu, Chunhui Hu, Xingquan Zhu, et al. A Unified Framework for Semantics and Feature Based Relevance Feedback in Image Retrieval Systems[C]. Proceedings of the eighth ACM international conference on Multimedia, Los Angeles, California, 2000:31-37.
[52] Xiaohang Ma,Dianhui Wang. Semantics Modeling Based Image Retrieval System Using Neural Networks[C], IEEE International Conference on Image Processing, 2005: I-1165-8.
[53] Xue Gao, Jian yu Chen. A Semantic Modeling Approach for Medical Image Semantic Retrieval Using Hybrid Bayesian Networks[C], Proceedings of the Sixth International Conference on Intelligent Systems Design and Applications, 2006: 482-487.
[54] Guixiong Liu, Xiongping Zhang. Multi-Class Classification of Support Vector Machines Based on Double Binary Tree[C], Fourth International Conference on Natural Computation, 2008:103-105.
[55] B. Fei and J. Liu, Binary tree of SVM: A new fast Multi-class Training and Classification Algorithm[C], IEEE Transactions on Neural Networks, vol.17, no.3, pp.696 -704, May 2006.
[56] J. Platt. Fast training of support vector machines using sequential minimal optimization[J], In Advances in Kernel Methods - Support Vector Learning, 1999:185-208.