基于内容的图像检索技术研究
|
摘要
随着计算机网络和多媒体技术的迅速发展,图像资源越来越丰富,传统的文本关键词检索方法已经不能满足图像信息的检索要求,基于内容的图像检索技术(Content-Based Image Retrieval,CBIR)应运而生,并成为图像领域研究的热点问题。本文重点探讨基于颜色特征和形状特征的图像检索方法。
首先,本文论述了目前国内外基于内容的图像检索系统的发展和研究现状,研究了基于内容的图像检索的关键技术,包括图像颜色、纹理、形状以及空间特征的提取,相似性度量,相关反馈机制和检索性能评价技术。
其次,对于形状特征的提取,提出了一种基于改进的不变矩特征的图像检索算法(ISE算法),在传统的七个Hu不变矩特征的基础上加入了离心率特征,把这八个特征作为图像的形状特征并进行归一化之后进行图像检索。实验数据表明,基于ISE算法的图像检索具有对图像的平移、旋转、缩放、扭曲的不变性,较好地保持了形状的一致性,尤其对于目标明确的图像具有良好的检索效果。
再次,对于颜色特征的提取,针对全局颜色直方图丢失了空间信息的缺点,对图像进行分块,并选用HSV颜色空间且将颜色量化为72维,统计每个分块的颜色直方图并选取像素点数目最多的那种颜色作为分块的主颜色。将示例图像与检索图像对应分块主颜色距离的累加和作为两幅图像的相似性距离进行图像的检索。
最后,研究了融合颜色和形状两种特征的图像检索方法,设计了一个基于分块主颜色和改进的形状不变矩特征的图像检索系统,并能够实现两种特征的单独检索功能和综合检索功能。实验结果表明,如果图像中的目标物体比较明显,或者想要检索到包含物体的平移、旋转、缩放以及形变的图像,使用形状不变矩特征的检索效果更好。用户可根据图像的特点给两种特征分配权值进行综合检索,比传统单一特征检索的效果要好。
With the development of network and multimedia technology, image resources become more and more abundant, the traditional method which based on text keyword is unable to fulfill the demand of image retrieval. Content-Based Image Retrieval which becomes one of the hottest research topics in the field of image has emerged at the right moment. This paper mainly investigated the methods of image retrieval which based on color and shape.
Firstly, a general overview of the development and research actualities of the content-based image retrieval system are introduced, and its key technology is researched, including the extraction of the image features (such as color, texture, shape and spatial relationship), the similarity measurement, relative feedback and retrieval performance evaluation.
Secondly, as for the extraction of the shape feature, an image retrieval algorithm (ISE) is proposed based on modified moment invariant features, in the algorithm, eccentricity is joined in the seven traditional Hu moment invariants. Experimental data show that image retrieval based on ISE algorithm has some advantages of translation, rotation, scaling and distortion invariance of image, and has good affect in maintaining the consistency of the shape, and especially has good retrieval results to the target specific images.
Thirdly, as for the color feature extraction, because of the loss of space information of global color histogram, image is divided into small pieces, HSV color space is chosen and the color is quantized to 72-dimensions. Color histogram of each partition is counted, and the color which contains the largest number of pixels is dominant color of the partition. The implementation of image retrieval is through counting similarity distance which is the cumulative distance of the corresponding blocks between the sample image and database images.
Finally, in order to design and implement the content-based image retrieval prototype system by synthesizing the feature of dominant color of the partition and modified shape invariant moment, the image retrieval method integrated with color and shape features are researched. The system can retrieval images based on single feature or color and shape features. The experimental results show that image retrieval based on invariant moment has better retrieval efficiency while the image has specific targets or the user want to retrieval images which include translation, rotation, scaling or distortion targets. According to the characteristics of the image, users can retrieval images by giving different ratio to the two features. The retrieval precision of fusion multi-feature is much higher than traditional single feature retrieval.
首先,本文论述了目前国内外基于内容的图像检索系统的发展和研究现状,研究了基于内容的图像检索的关键技术,包括图像颜色、纹理、形状以及空间特征的提取,相似性度量,相关反馈机制和检索性能评价技术。
其次,对于形状特征的提取,提出了一种基于改进的不变矩特征的图像检索算法(ISE算法),在传统的七个Hu不变矩特征的基础上加入了离心率特征,把这八个特征作为图像的形状特征并进行归一化之后进行图像检索。实验数据表明,基于ISE算法的图像检索具有对图像的平移、旋转、缩放、扭曲的不变性,较好地保持了形状的一致性,尤其对于目标明确的图像具有良好的检索效果。
再次,对于颜色特征的提取,针对全局颜色直方图丢失了空间信息的缺点,对图像进行分块,并选用HSV颜色空间且将颜色量化为72维,统计每个分块的颜色直方图并选取像素点数目最多的那种颜色作为分块的主颜色。将示例图像与检索图像对应分块主颜色距离的累加和作为两幅图像的相似性距离进行图像的检索。
最后,研究了融合颜色和形状两种特征的图像检索方法,设计了一个基于分块主颜色和改进的形状不变矩特征的图像检索系统,并能够实现两种特征的单独检索功能和综合检索功能。实验结果表明,如果图像中的目标物体比较明显,或者想要检索到包含物体的平移、旋转、缩放以及形变的图像,使用形状不变矩特征的检索效果更好。用户可根据图像的特点给两种特征分配权值进行综合检索,比传统单一特征检索的效果要好。
With the development of network and multimedia technology, image resources become more and more abundant, the traditional method which based on text keyword is unable to fulfill the demand of image retrieval. Content-Based Image Retrieval which becomes one of the hottest research topics in the field of image has emerged at the right moment. This paper mainly investigated the methods of image retrieval which based on color and shape.
Firstly, a general overview of the development and research actualities of the content-based image retrieval system are introduced, and its key technology is researched, including the extraction of the image features (such as color, texture, shape and spatial relationship), the similarity measurement, relative feedback and retrieval performance evaluation.
Secondly, as for the extraction of the shape feature, an image retrieval algorithm (ISE) is proposed based on modified moment invariant features, in the algorithm, eccentricity is joined in the seven traditional Hu moment invariants. Experimental data show that image retrieval based on ISE algorithm has some advantages of translation, rotation, scaling and distortion invariance of image, and has good affect in maintaining the consistency of the shape, and especially has good retrieval results to the target specific images.
Thirdly, as for the color feature extraction, because of the loss of space information of global color histogram, image is divided into small pieces, HSV color space is chosen and the color is quantized to 72-dimensions. Color histogram of each partition is counted, and the color which contains the largest number of pixels is dominant color of the partition. The implementation of image retrieval is through counting similarity distance which is the cumulative distance of the corresponding blocks between the sample image and database images.
Finally, in order to design and implement the content-based image retrieval prototype system by synthesizing the feature of dominant color of the partition and modified shape invariant moment, the image retrieval method integrated with color and shape features are researched. The system can retrieval images based on single feature or color and shape features. The experimental results show that image retrieval based on invariant moment has better retrieval efficiency while the image has specific targets or the user want to retrieval images which include translation, rotation, scaling or distortion targets. According to the characteristics of the image, users can retrieval images by giving different ratio to the two features. The retrieval precision of fusion multi-feature is much higher than traditional single feature retrieval.
引文
[1]王文惠,周良柱,万建伟.基于内容的图象检索技术的研究和发展[J].计算机工程与应用, 2001. 5: 54-56.
[2]黄祥林,沈兰荪.基于内容的图像检索技术研究[J].电子学报, 2002, 30(7): 1066-1071.
[3]李向阳,庄越挺,潘云鹤.基于内容的图像检索技术与系统[J].计算机研究与发展, 2001, 38(3): 344-354.
[4]王文惠,周良柱,万建伟.基于内容的图像检索技术的研究和发展.计算机工程与应用, 2001, 37(5): 54-55, 66.
[5] H. Tamura and N. Yokoya,“Image database systems: A survey,”Pattern Recognition, 17, 1984, 29-43.
[6] S. K. Chang and A. Hsu,“Image information systems: Where do we go from here?”, IEEE Transactions on Knowledge and Data Engineering, 4, 1992, 431-442.
[7]田浩,葛秀慧,王顶等译.数字图像处理.清华大学出版社, 2007, 167-174.
[8]李向阳.基于内容的图像数据库检索技术及其模型的研究[D].浙江:浙江大学, 1999.
[9] J. Dowe. Content-based retrieval in multimedia imaging[C]. In Proc. SPIE Storage and Retrieval for Image and Video Database, 1993.
[10] C. Faloutsos et a1. Efficient and effective querying by image content[J]. Journal of intelligent information systems, Vol. 3, 1994: 231-262.
[11]温泉彻,彭宏,黎琼.基于内容的图像检索关键技术研究[J].微计算机信息(管控一体化), 2007年第23卷: 278.
[12]杭燕,杨育彬等.基于内容的图像检索综述.计算机应用研究, 2002,19(3):9-13,29.
[13] Smith J R, Chang S F. VirualSeek: AFully Automated Content-Based Image Query System. ACM Multimedia 96, Boston, MA, Nov.
[14]朱兴全,张宏江,刘文印,吴立德. iFind:一个结合语义和视觉特征的图像相关反馈检索系统[J].计算机学报, 2002年07期: 10-17.
[15] Zhang HJ, Lju W, Hu C. A System for Semantics and Feature Based Image Retrieval over Internet.Proc. 8 ACM Multimedia Coaf, Los Angeles, USA, 2000. 477-478.
[16] B. S. Manjunath, Jens-Rainer Ohm. Color and Texture Descriptor. Circuits and Systems for VideoTechnology, 2001(7): 24-30.
[17]王洪君.基于内容的图像检索技术.松辽学刊(自然科学版), 2002, 5(2): 4-6.
[18]徐杰,施鹏飞.基于内容的图像检索技术.中国图像图形学报, 2003, 8(9):977-983.
[19] M. Swain and D. Ballard,“Color indexing,”International Journal of Computer Vision, 1991, 11-32.
[20] M. Strickeer and M. Orengo,“Similarity of color images,”in Proceedings of SPIE Storage and Retrieval for Image and Video Databases III, vol. 2185(San Jose, CA), February 1995, 381-392.
[21]施智平,高光来,赵晓春等.一个基于颜色特征的图像检索系统.内蒙古大学学报(自然科学版), 2003, 34(2): 226-230.
[22] G. Pass and R. Zabith,“Histogram refinement for content-based image retrieval,”in Proceedings of IEEE Workshop and Applications of Computer Vision, 1996, 96-102.
[23]韩军伟.基于内容的图像检索技术研究[D].西北工业大学, 2003.
[24] R M Haralick. Statistical and Structural Approaches to Texture [J]. Proc. IEEE, 1979, 67: 786-804.
[25] H. Tamura, S. Mori, and T. Yamawaki,“Texture feature for image classification,”IEEE Transactions on Systems, Man, and Cybernetics, 3, 1973, 610-621.
[26]章毓晋著.基于内容的视觉信息检索.北京:科学出版社, 2003.
[27] Milan Sonka, Vaclay Hlavac, Roger Boyle著.艾海舟,武勃等译.图像处理、分析与机器视觉(第二版).北京:人民邮电出版社, 2003.
[28] Del Bimbo A, Pah P. Shape indexing by multi-scale representation. Image and vision computing, 1999, 17, 245-26l.
[29] Tversky A. Feature ofsimilafity. Psychological review, 1977, 84(4): 327-352.
[30] A. Franco, A. Lumini, D. Maio. A New Apporach for Relevance Feedback Through Positive and Negative Samples. ICPR(4)2004: 905-908.
[31] Q. Iqbal and JK Aggarwal. Feature Integration, multi-image queries and relevance feedback in image retrieval. 6th international conference on virsual information systems (VISUAL), 2003, 467-474.
[32] Zhong Su, Hongjiang Zhang, Shaoping Ma. Using Bayesian classifier in relevant feedback of image retrieval[A]. in: Proceedings of 12th IEEE International Conference on Tools with Artificial Intelligence, 2000, 258-261.
[33] Rui Y. Huang T S. Mehrotra S. Ortega M. Relevalce feedback: A Power tool for interactive content-based image retrieval[J]. IEEE Trans. Circuits and Systems for Video Technology. 1998, 8(5): 644-655.
[34] Y. Rui, T. S. Huang, and S. F. Chang. Image retrieval: Past, present and future[J]. Journal of Visual Communication and Image Representation, 1990, (10): 1-23.
[35]韦娜,耿国华,周明全.基于内容的图像检索系统性能评价[J].中国图像图形学报, 2004, 9(11): 1271-1276.
[36] B. M. Mehtre, et a1. Colour matching for image retrieval[J]. Pattern Recognition Letters, 1995, 16: 324-331.
[37] Jain A K, Vailaya A. Image Retrieval Using Color and Shape[J]. Pattern Recognition, 1996, 29(8): 1233-1244.
[38] Zahn C T, Roskies R Z. Fourier Descriptors for Plane Closed Curves[J]. IEEE Transactions on Computers, 1972, (21): 269-285.
[39] Rui Y, et al. Modified Fourier Descriptor for Shape Representation: A Practical Approach [C]. Proc. 1st International Workshop on Image Database and Multimedia Search, Amsterdam, the Netherlands, 1996.
[40] Liu W Y, Wang T, Zhang H J. A Hierarchical Characterization Scheme for Image Retrieval[C]. Proc. 7th IEEE International Conference on Image Processing (ICIP 2000), Vancouver, Canada, 2000.
[41] Chen Y X, Wang J Z. A Region-based Fuzzy Feature Matching Approach to Content-based Image Retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(9): 1252-1267.
[42] Jain A K, Vailaya A. Shape-based Retrieval: A Case Study with Trademark Image Databases[J]. Pattern Recognition, 1998, 31(9): 1369-1390.
[43] Hans A Hegt, et al. A High Performance License Plate Recognition System[C]. IEEE International Conference on Systems, Man, and Cybernetics, 1998, 4357-4362.
[44] Rafael C. Gonzalez, Richard E. Woods著.阮秋琦,阮宇智等译.数字图像处理(第二版).电子工业出版社, 2003. 3, 545-549.
[45] Translated by TIAN Hao, GE Xiu-hui, WANG Ding, et al. Digital Image Processing[M]. Tsinghua University Press, 2007.
[46]刘芳,王改梅.综合颜色特征的彩色图像检索方法.计算机工程与应用, 2003, 16: 83-85.
[47]阮秋琦著.数字图像处理学.电子工业出版社, 2007. 2, 221-228.
[48]刘忠伟,章毓晋.十种基于颜色特征的图像检索算法的比较和分析[J].信号处理, 2000, 16(1): 79-94.
[49]薛少娟,左万利,赫枫龄.基于颜色分块全局直方图的图像检索方法及系统实现.吉林大学学报(理学版), 2006, 44(4): 608-612.
[50] http://wenku.baidu.com/view/290c289851e79b896802267e.html.
[2]黄祥林,沈兰荪.基于内容的图像检索技术研究[J].电子学报, 2002, 30(7): 1066-1071.
[3]李向阳,庄越挺,潘云鹤.基于内容的图像检索技术与系统[J].计算机研究与发展, 2001, 38(3): 344-354.
[4]王文惠,周良柱,万建伟.基于内容的图像检索技术的研究和发展.计算机工程与应用, 2001, 37(5): 54-55, 66.
[5] H. Tamura and N. Yokoya,“Image database systems: A survey,”Pattern Recognition, 17, 1984, 29-43.
[6] S. K. Chang and A. Hsu,“Image information systems: Where do we go from here?”, IEEE Transactions on Knowledge and Data Engineering, 4, 1992, 431-442.
[7]田浩,葛秀慧,王顶等译.数字图像处理.清华大学出版社, 2007, 167-174.
[8]李向阳.基于内容的图像数据库检索技术及其模型的研究[D].浙江:浙江大学, 1999.
[9] J. Dowe. Content-based retrieval in multimedia imaging[C]. In Proc. SPIE Storage and Retrieval for Image and Video Database, 1993.
[10] C. Faloutsos et a1. Efficient and effective querying by image content[J]. Journal of intelligent information systems, Vol. 3, 1994: 231-262.
[11]温泉彻,彭宏,黎琼.基于内容的图像检索关键技术研究[J].微计算机信息(管控一体化), 2007年第23卷: 278.
[12]杭燕,杨育彬等.基于内容的图像检索综述.计算机应用研究, 2002,19(3):9-13,29.
[13] Smith J R, Chang S F. VirualSeek: AFully Automated Content-Based Image Query System. ACM Multimedia 96, Boston, MA, Nov.
[14]朱兴全,张宏江,刘文印,吴立德. iFind:一个结合语义和视觉特征的图像相关反馈检索系统[J].计算机学报, 2002年07期: 10-17.
[15] Zhang HJ, Lju W, Hu C. A System for Semantics and Feature Based Image Retrieval over Internet.Proc. 8 ACM Multimedia Coaf, Los Angeles, USA, 2000. 477-478.
[16] B. S. Manjunath, Jens-Rainer Ohm. Color and Texture Descriptor. Circuits and Systems for VideoTechnology, 2001(7): 24-30.
[17]王洪君.基于内容的图像检索技术.松辽学刊(自然科学版), 2002, 5(2): 4-6.
[18]徐杰,施鹏飞.基于内容的图像检索技术.中国图像图形学报, 2003, 8(9):977-983.
[19] M. Swain and D. Ballard,“Color indexing,”International Journal of Computer Vision, 1991, 11-32.
[20] M. Strickeer and M. Orengo,“Similarity of color images,”in Proceedings of SPIE Storage and Retrieval for Image and Video Databases III, vol. 2185(San Jose, CA), February 1995, 381-392.
[21]施智平,高光来,赵晓春等.一个基于颜色特征的图像检索系统.内蒙古大学学报(自然科学版), 2003, 34(2): 226-230.
[22] G. Pass and R. Zabith,“Histogram refinement for content-based image retrieval,”in Proceedings of IEEE Workshop and Applications of Computer Vision, 1996, 96-102.
[23]韩军伟.基于内容的图像检索技术研究[D].西北工业大学, 2003.
[24] R M Haralick. Statistical and Structural Approaches to Texture [J]. Proc. IEEE, 1979, 67: 786-804.
[25] H. Tamura, S. Mori, and T. Yamawaki,“Texture feature for image classification,”IEEE Transactions on Systems, Man, and Cybernetics, 3, 1973, 610-621.
[26]章毓晋著.基于内容的视觉信息检索.北京:科学出版社, 2003.
[27] Milan Sonka, Vaclay Hlavac, Roger Boyle著.艾海舟,武勃等译.图像处理、分析与机器视觉(第二版).北京:人民邮电出版社, 2003.
[28] Del Bimbo A, Pah P. Shape indexing by multi-scale representation. Image and vision computing, 1999, 17, 245-26l.
[29] Tversky A. Feature ofsimilafity. Psychological review, 1977, 84(4): 327-352.
[30] A. Franco, A. Lumini, D. Maio. A New Apporach for Relevance Feedback Through Positive and Negative Samples. ICPR(4)2004: 905-908.
[31] Q. Iqbal and JK Aggarwal. Feature Integration, multi-image queries and relevance feedback in image retrieval. 6th international conference on virsual information systems (VISUAL), 2003, 467-474.
[32] Zhong Su, Hongjiang Zhang, Shaoping Ma. Using Bayesian classifier in relevant feedback of image retrieval[A]. in: Proceedings of 12th IEEE International Conference on Tools with Artificial Intelligence, 2000, 258-261.
[33] Rui Y. Huang T S. Mehrotra S. Ortega M. Relevalce feedback: A Power tool for interactive content-based image retrieval[J]. IEEE Trans. Circuits and Systems for Video Technology. 1998, 8(5): 644-655.
[34] Y. Rui, T. S. Huang, and S. F. Chang. Image retrieval: Past, present and future[J]. Journal of Visual Communication and Image Representation, 1990, (10): 1-23.
[35]韦娜,耿国华,周明全.基于内容的图像检索系统性能评价[J].中国图像图形学报, 2004, 9(11): 1271-1276.
[36] B. M. Mehtre, et a1. Colour matching for image retrieval[J]. Pattern Recognition Letters, 1995, 16: 324-331.
[37] Jain A K, Vailaya A. Image Retrieval Using Color and Shape[J]. Pattern Recognition, 1996, 29(8): 1233-1244.
[38] Zahn C T, Roskies R Z. Fourier Descriptors for Plane Closed Curves[J]. IEEE Transactions on Computers, 1972, (21): 269-285.
[39] Rui Y, et al. Modified Fourier Descriptor for Shape Representation: A Practical Approach [C]. Proc. 1st International Workshop on Image Database and Multimedia Search, Amsterdam, the Netherlands, 1996.
[40] Liu W Y, Wang T, Zhang H J. A Hierarchical Characterization Scheme for Image Retrieval[C]. Proc. 7th IEEE International Conference on Image Processing (ICIP 2000), Vancouver, Canada, 2000.
[41] Chen Y X, Wang J Z. A Region-based Fuzzy Feature Matching Approach to Content-based Image Retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(9): 1252-1267.
[42] Jain A K, Vailaya A. Shape-based Retrieval: A Case Study with Trademark Image Databases[J]. Pattern Recognition, 1998, 31(9): 1369-1390.
[43] Hans A Hegt, et al. A High Performance License Plate Recognition System[C]. IEEE International Conference on Systems, Man, and Cybernetics, 1998, 4357-4362.
[44] Rafael C. Gonzalez, Richard E. Woods著.阮秋琦,阮宇智等译.数字图像处理(第二版).电子工业出版社, 2003. 3, 545-549.
[45] Translated by TIAN Hao, GE Xiu-hui, WANG Ding, et al. Digital Image Processing[M]. Tsinghua University Press, 2007.
[46]刘芳,王改梅.综合颜色特征的彩色图像检索方法.计算机工程与应用, 2003, 16: 83-85.
[47]阮秋琦著.数字图像处理学.电子工业出版社, 2007. 2, 221-228.
[48]刘忠伟,章毓晋.十种基于颜色特征的图像检索算法的比较和分析[J].信号处理, 2000, 16(1): 79-94.
[49]薛少娟,左万利,赫枫龄.基于颜色分块全局直方图的图像检索方法及系统实现.吉林大学学报(理学版), 2006, 44(4): 608-612.
[50] http://wenku.baidu.com/view/290c289851e79b896802267e.html.