基于相关反馈的图像检索研究
|
摘要
为了更好地解决基于内容的图像检索中低级图像视觉特征和高级人类语言之间的语义鸿沟问题,在检索基本模块中加入相关反馈的模块,让用户参与到检索过程中,提交对于前次检索的意见帮助机器学习,更进一步了解用户的检索需求,更可能准确地“猜测”用户意图。
本文在构建相关反馈的检索模型时,对图像进行分块提取特征向量,使用直方图提取颜色特征,使用基于MPEG-7的边缘直方图描述符提取纹理特征,使用多元正态分布来建模特征空间,使用LOGISTIC回归模型对图像特征向量的内部向量进行动态权值的调整,使用新的权值计算图像间相似度,使用贝叶斯估计模型估算出调整权值后图像数据库中图像符合用户前一次标记特点的概率,以此为排序基础对图像进行降序输出。主要研究工作有：
1.学习了现有的图像特征提取方法,特别是颜色和纹理特征的提取算法,根据人眼对图像颜色和纹理的视觉感知敏感度的不同,提出分配两种特征不同维度的提取方法,更高效率地利用特征向量的维度。
2.通过研究动态调整特征权值的相关内容,选择LOGISTIC回归模型计算特征向量内部向量的动态权值。
3.进行过权值调整后采用贝叶斯估计模型,计算预测概率,用预测概率取代相似度作为输出图像的顺序。
4.在matlab平台下提取特征,在Java环境下搭建仿真系统,经大量实验证明此方法有较好的查全率和查准率。平均查全率为0.80,平均查准率为0.85。
To solve the semantic gap problem between low-level visual features in images and high-level human language in content-based image retrieval system better, it makes users easier to participate into the retrieval process by adding relevance feedback module to the basic retrieval module. When uses present judgment for previous retrieval result to help machine learning and better understanding the users' intention, it will guess the intention more accurately.
In this relevance feedback model, the feature vectors are extracted in five sub-blocks, the color features are extracted by histogram, the texture features are extracted by using edge histogram descriptor based on MPEG-7, the feature space is builded with multivariate normal distribution, the dynamic weights of the internal vectors are reckoned using LOGISTIC regression model, and finally Bayesian posterior is used to estimate posterior probabilities of all images which shows how the retrieved images after adjusting weights meet the users' requirement. Correlation images are outputted according to the probabilities in descending order. The main research works are:
1. Learning the current image feature extraction methods, especially color and texture feature extraction algorithm, extracting two different dimensions of two kind of features is more efficient.
2. The LOGISTIC regression model used to calculate the dynamic weight of the initial feature vectors is choosen after searching for relevant content about dynamic adjustment of feature weights.
3. After adjusting the weights, using Bayesian estimation model to calculate the predicted probabilities, images are exported on the basis of predicted probabilities instead of the sequence similarities.
4. The experiments are done by matlab and Java softwares and the results show that this method has better recall and precision. The average recall is 0.80 and the average precision is 0.85.
本文在构建相关反馈的检索模型时,对图像进行分块提取特征向量,使用直方图提取颜色特征,使用基于MPEG-7的边缘直方图描述符提取纹理特征,使用多元正态分布来建模特征空间,使用LOGISTIC回归模型对图像特征向量的内部向量进行动态权值的调整,使用新的权值计算图像间相似度,使用贝叶斯估计模型估算出调整权值后图像数据库中图像符合用户前一次标记特点的概率,以此为排序基础对图像进行降序输出。主要研究工作有：
1.学习了现有的图像特征提取方法,特别是颜色和纹理特征的提取算法,根据人眼对图像颜色和纹理的视觉感知敏感度的不同,提出分配两种特征不同维度的提取方法,更高效率地利用特征向量的维度。
2.通过研究动态调整特征权值的相关内容,选择LOGISTIC回归模型计算特征向量内部向量的动态权值。
3.进行过权值调整后采用贝叶斯估计模型,计算预测概率,用预测概率取代相似度作为输出图像的顺序。
4.在matlab平台下提取特征,在Java环境下搭建仿真系统,经大量实验证明此方法有较好的查全率和查准率。平均查全率为0.80,平均查准率为0.85。
To solve the semantic gap problem between low-level visual features in images and high-level human language in content-based image retrieval system better, it makes users easier to participate into the retrieval process by adding relevance feedback module to the basic retrieval module. When uses present judgment for previous retrieval result to help machine learning and better understanding the users' intention, it will guess the intention more accurately.
In this relevance feedback model, the feature vectors are extracted in five sub-blocks, the color features are extracted by histogram, the texture features are extracted by using edge histogram descriptor based on MPEG-7, the feature space is builded with multivariate normal distribution, the dynamic weights of the internal vectors are reckoned using LOGISTIC regression model, and finally Bayesian posterior is used to estimate posterior probabilities of all images which shows how the retrieved images after adjusting weights meet the users' requirement. Correlation images are outputted according to the probabilities in descending order. The main research works are:
1. Learning the current image feature extraction methods, especially color and texture feature extraction algorithm, extracting two different dimensions of two kind of features is more efficient.
2. The LOGISTIC regression model used to calculate the dynamic weight of the initial feature vectors is choosen after searching for relevant content about dynamic adjustment of feature weights.
3. After adjusting the weights, using Bayesian estimation model to calculate the predicted probabilities, images are exported on the basis of predicted probabilities instead of the sequence similarities.
4. The experiments are done by matlab and Java softwares and the results show that this method has better recall and precision. The average recall is 0.80 and the average precision is 0.85.
引文
[1].Liu Y, Zhang DS, Lu GJ, et al. A survey of content-based image retrieval with high-level semantics[J]. The Journal of the Pattern Recognition Society,2007,40(1):262-282.
[2]D.J.van Rijsbergen. Information Retrieval[M]. London:Butter-worths,1979.
[3]R.Picard, T.P.Minka, M.Szummer. Modeling User Subjectivity in Image Libraries[R]. Switzerland:In Proceedings of International Conference on Image Processing,1996.
[4]庄越挺,潘云鹤,Thomas S.Huang.基于内容的图像检索综述[J].模式识别与人工智能,1999,12(2)：170-177.
[5]Finn R. Query by image content[R]. New York:IBM Research Center,1996.
[6]Ingemar J.Cox, Matt L.Miller, Thomas P.Minka, et al. The Bayesian image retrieval system, picHunter, theory, implementation and pychophysical experiments[J]. IEEE Transactions Image Processing,2000,9(1):20-37.
[7]Carlton W.Niblack, Ron Barber, Eill Equitz, et al. The QBIC project:Querying images by content using color, texture and shape[R]. San Jose:The International Society for Optical Enginerring Conference on Storage and Retrieval for Image and Video Database,1993.
[8]Bach J R, Fuller C, Gupta A, et al. Virage image search engine:an open framework for image management[R]. San Jose:The International Society for Optical Enginerring Conference Storage and Retrieval for Image and Video Databases IV,1996.
[9]John R. Smith, Shih-Fu Chang. VisualSeek:a fully automated content-based image query system[R]. Boston:In Processing of ACM International Conference Multimedia,1996.
[10]Pentland A. Photobook:tools for content-based manipulation of image databases[J]. International Journal of Computer Vision,1996,18(3):233-254.
[11]Wu Hong, Lu HQ, Ma SD. Multilevel Relevance Judgment, Loss Function and Performance Measure in Image Retrieval[R]. Urbana:In Proceedings of International Conference on Image and Video Retrieval,2003.
[12]Szummer, M.Picard, R.W. Indoor-outdoor image classification[R]. Bombay:In Proceedings of the IEEE International Workshop on Content-based Access of Image and Video Database,1998.
[13]John P.Eakins. Automatic image content-retrieval are we getting anywhere[R]. Milton Keynes:In Proceedings of Third International Conference on Electronic Library and Visual Information Research,1996.
[14]Shih-Fu Chang, William Chen, Hari Sundaram. Semantic visual templates, Linking visual features to semantics[R]. Illinois:In Proceedings of the IEEE International Conference on Image Processing,1998.
[15]John R. Smith, Shih-Fu Chang. Automated Image Retrieval Using Color and Texture[R]. New York:IEEE Transactions on Pattern Analysis and Machine Intelligence,1996.
[16]冯亚,耿国华,周明全等.基于颜色特征图像检索与相关反馈综合研究[J].计算机技术与发展,2007,17(12)：251-254.
[17]Yong Rui, Thomas S.Huang, Shih-Fu Chang. Image Retrieval:Past, Present and Future[R]. Taipei:In Proceedings of International Symposium on Multimedia Information Processing,1997.
[18]Greg.Pass, Ramin.Zabih, Justin.Miller. Comparing Images Using Color Coherence Vectors[R]. Boston:In Proceedings of Association for ComputingMachinery Multimedia, 1996.
[19]Donald Hearn, M.Pauline Baker. Computer Graphics[M]. New York:Prentice Hall International,1994.
[20]Todd R.Reed, J.M.Hans du Buf. A review of recent texture segmentation and feature extraction technique[J]. Graphical Models and Image Processing Image Understanding, 1993,57(3):359-372.
[21]Raphael Gonzalez, Richard E. Digital Image Processing,2nd Ed[M]. New York: Prentice Hall Press,2002.
[22]孟繁杰,郭宝龙.CBIR关键技术研究[J].计算机应用研究,2004,21(7)：21-24.
[23]耿国华,周明全.常用色彩量化算法的性能分析[J].小型微型计算机系统,1998,19(9)：46-49.
[24]毛力,张晓林.基于颜色内容的图像检索原理与方法[J].情报科学,2000,18(6)：552-555.
[25]Tamura, Mori H., Yamawaki S, et al. Textural features corresponding to visual perception[J]. IEEE Transactions on Systems, Man and Cybernetics,1978,8(6):460-473.
[26]John R.Smith, Shih-Fu Chang. Transform Features for Texture Classification and Discrimination in Large Image Databases[R]. New York:In Proceedings of IEEE International Conference on Image Processing,1994.
[27]Chuang GH, Kuo CJ. Wavelet Descriptor of Planar Curves:Theory and Applications[J]. IEEE Transaction on Image Processing,1996,5(1):56-70.
[28]谢昌平,孙劲光,崔彩峰.一种基于轮廓的图像检索算法研究[J].宁波大学学报(理工版),2007,12(20)：438-440.
[29]Li ZM, Hou KP, Liu YJ, et al. The Shape Recognition Based on Structure Moment Invariants[J]. International Journal of Information Technology,2006,12(2):97-105.
[30]Yong Rui, Thomas S.Huang. Content-Based Image Retrieval with Relevance Feedback in MARS[R]. New York:In Proceedings of IEEE International Conference on Image Processing, 1997.
[31]曹奎,冯玉才,曹忠升.基于颜色和形状特征的彩色图像表示与检索技术[J].计算机辅助设计与图形学学报,2001,13(10)：907-911.
[32]Feng Jing, Li MJ, Zhang HJ, et al. A Unified Framework for Image Retrieval Using Keyword and Visual Features[J]. IEEE Transaction on Image Processing,2005,14(7): 979-989.
[33]吴洪,卢汉清,马颂德.基于内容图像检索中相关反馈技术的回顾[J].计算机学报,2005,12(10)：25-27.
[34]邸东泉,萧宝瑾.基于移动中心点图像检索的相关反馈技术[J].中国新技术新产品,2009,16(3)：9.
[35]I.Bartolini, P.Ciaccia, M.Patella. WARP:Accurate Retrieval of Shapes Using Phase of Fourier Descriptors and Time Warping Distance[J]. IEEE Transaction on Pattern Analysis and Machine Intelligence,2005,27(3):142-147.
[36]David A.White, Ramesh Jain. Similarity Indexing with the SS-tree[R]. New Orleans:In Proceedings of IEEE International Conference on Data Engineering,1996.
[37]Baback Moghaddam, Qi.Tian, Thomas S.Huang. Spatial Visualization for Content-Based Image Retrieval[R]. Tokyo:In Proceedings of IEEE International Conference on Multimedia and Expo,2001.
[38]张磊,林福宗,张跋.基于支持向量机的相关反馈图像检索算法[J].清华大学学报(自然科学版),2003,42(1)：58-63.
[39]Feng Jing, Li MJ. An Effective Region-Based Image Retrieval Framework[J]. IEEE Transactions on Image Processing,2004,13(5):699-709.
[40]He DA, Nick Cercone, Gu ZM. Applying the Extended Mass-Constraint EM Algorighm to Image Retrieval[J]. Computers and Mathematics with Applications,2008,56(4): 1010-1024.
[41]Han JW, Ngan King N., Li MJ, et al. A Memory Learning Framework for Effective Image Retrieval[J]. IEEE Trans. Image Processing,2005,14(4):511-524.
[42]Yong Rui, Thomas S.Huang, Mehrotra S. Relevance Feedback:A Powerful Tool in Interactive Content-based Image Retrieval[J]. IEEE Transactions on Image Processing,1998, 28(5):644-655.
[43]王济川,郭志刚.LOGISTIC回归模型方法与应用[M].北京：高等教育出版社,2001：14-18.
[44]James O.Ramsay, B.W.Silverman. Functional Data Analysis[M]. New York:Springer Series in Statistics,1997.
[45]Claudia C. On selecting parametric link transformation families in generalized linear models[J]. Journal of Statistical Planning and Inference,1997,61(6):125-139.
[46]陈希孺.广义线性模型(一)[J].数理统计与管理,2002,21(5)：54-61.
[47]丁洁丽,陈希孺.广义线性模型极大似然估计的大样本理论[D].武汉大学：武汉大学概率论与数理统计,2006.
[48]王济川,谢海义,姜宝法.多层统计分析模型[M].北京：高等教育出版社,2008.
[49]茆诗松.贝叶斯统计[M].北京：中国统计出版社,1999：35-48.
[50]朱慧明,韩玉启.贝叶斯多元统计推断理论[M].北京：科学出版社,2006：41-45.
[51]Ingemar J.Cox, Matt L.Miller, Stephen M., et al. Bayesian Relevance Feedback for Image Retrieval[R]. Austria:International Conference on Pattern Recognition,1996.
[52]Su Zhong, Zhang HJ, Stan Li, et al. Relevance Feedback in Content-based Image Retrieval:Bayesian Framework,Feature Subspaces,and Progressive learning[J]. IEEE Transaction on Image Processing,2003,12(8):924-937.
[53]Nuno Vasconcelos, Andrew Lippman. Bayesian Relevance Feedback for Content-based Image Retrieval[R]. Washiongton:In Proceedings of IEEE Workshop on Content-based Access to Image and Video Library,2000.
[54]Nastar C, Meilhac C. Relevance Feedback and Category Search in Image Databases[R]. Florence:International Conference on Multimedia Computing and Systems,1999.
[55]赵玉凤,赵耀,朱振峰.贝叶斯框架下基于区域的相关反馈算法[J].电子与信息学报,2008,30(4)：937-940.
[2]D.J.van Rijsbergen. Information Retrieval[M]. London:Butter-worths,1979.
[3]R.Picard, T.P.Minka, M.Szummer. Modeling User Subjectivity in Image Libraries[R]. Switzerland:In Proceedings of International Conference on Image Processing,1996.
[4]庄越挺,潘云鹤,Thomas S.Huang.基于内容的图像检索综述[J].模式识别与人工智能,1999,12(2)：170-177.
[5]Finn R. Query by image content[R]. New York:IBM Research Center,1996.
[6]Ingemar J.Cox, Matt L.Miller, Thomas P.Minka, et al. The Bayesian image retrieval system, picHunter, theory, implementation and pychophysical experiments[J]. IEEE Transactions Image Processing,2000,9(1):20-37.
[7]Carlton W.Niblack, Ron Barber, Eill Equitz, et al. The QBIC project:Querying images by content using color, texture and shape[R]. San Jose:The International Society for Optical Enginerring Conference on Storage and Retrieval for Image and Video Database,1993.
[8]Bach J R, Fuller C, Gupta A, et al. Virage image search engine:an open framework for image management[R]. San Jose:The International Society for Optical Enginerring Conference Storage and Retrieval for Image and Video Databases IV,1996.
[9]John R. Smith, Shih-Fu Chang. VisualSeek:a fully automated content-based image query system[R]. Boston:In Processing of ACM International Conference Multimedia,1996.
[10]Pentland A. Photobook:tools for content-based manipulation of image databases[J]. International Journal of Computer Vision,1996,18(3):233-254.
[11]Wu Hong, Lu HQ, Ma SD. Multilevel Relevance Judgment, Loss Function and Performance Measure in Image Retrieval[R]. Urbana:In Proceedings of International Conference on Image and Video Retrieval,2003.
[12]Szummer, M.Picard, R.W. Indoor-outdoor image classification[R]. Bombay:In Proceedings of the IEEE International Workshop on Content-based Access of Image and Video Database,1998.
[13]John P.Eakins. Automatic image content-retrieval are we getting anywhere[R]. Milton Keynes:In Proceedings of Third International Conference on Electronic Library and Visual Information Research,1996.
[14]Shih-Fu Chang, William Chen, Hari Sundaram. Semantic visual templates, Linking visual features to semantics[R]. Illinois:In Proceedings of the IEEE International Conference on Image Processing,1998.
[15]John R. Smith, Shih-Fu Chang. Automated Image Retrieval Using Color and Texture[R]. New York:IEEE Transactions on Pattern Analysis and Machine Intelligence,1996.
[16]冯亚,耿国华,周明全等.基于颜色特征图像检索与相关反馈综合研究[J].计算机技术与发展,2007,17(12)：251-254.
[17]Yong Rui, Thomas S.Huang, Shih-Fu Chang. Image Retrieval:Past, Present and Future[R]. Taipei:In Proceedings of International Symposium on Multimedia Information Processing,1997.
[18]Greg.Pass, Ramin.Zabih, Justin.Miller. Comparing Images Using Color Coherence Vectors[R]. Boston:In Proceedings of Association for ComputingMachinery Multimedia, 1996.
[19]Donald Hearn, M.Pauline Baker. Computer Graphics[M]. New York:Prentice Hall International,1994.
[20]Todd R.Reed, J.M.Hans du Buf. A review of recent texture segmentation and feature extraction technique[J]. Graphical Models and Image Processing Image Understanding, 1993,57(3):359-372.
[21]Raphael Gonzalez, Richard E. Digital Image Processing,2nd Ed[M]. New York: Prentice Hall Press,2002.
[22]孟繁杰,郭宝龙.CBIR关键技术研究[J].计算机应用研究,2004,21(7)：21-24.
[23]耿国华,周明全.常用色彩量化算法的性能分析[J].小型微型计算机系统,1998,19(9)：46-49.
[24]毛力,张晓林.基于颜色内容的图像检索原理与方法[J].情报科学,2000,18(6)：552-555.
[25]Tamura, Mori H., Yamawaki S, et al. Textural features corresponding to visual perception[J]. IEEE Transactions on Systems, Man and Cybernetics,1978,8(6):460-473.
[26]John R.Smith, Shih-Fu Chang. Transform Features for Texture Classification and Discrimination in Large Image Databases[R]. New York:In Proceedings of IEEE International Conference on Image Processing,1994.
[27]Chuang GH, Kuo CJ. Wavelet Descriptor of Planar Curves:Theory and Applications[J]. IEEE Transaction on Image Processing,1996,5(1):56-70.
[28]谢昌平,孙劲光,崔彩峰.一种基于轮廓的图像检索算法研究[J].宁波大学学报(理工版),2007,12(20)：438-440.
[29]Li ZM, Hou KP, Liu YJ, et al. The Shape Recognition Based on Structure Moment Invariants[J]. International Journal of Information Technology,2006,12(2):97-105.
[30]Yong Rui, Thomas S.Huang. Content-Based Image Retrieval with Relevance Feedback in MARS[R]. New York:In Proceedings of IEEE International Conference on Image Processing, 1997.
[31]曹奎,冯玉才,曹忠升.基于颜色和形状特征的彩色图像表示与检索技术[J].计算机辅助设计与图形学学报,2001,13(10)：907-911.
[32]Feng Jing, Li MJ, Zhang HJ, et al. A Unified Framework for Image Retrieval Using Keyword and Visual Features[J]. IEEE Transaction on Image Processing,2005,14(7): 979-989.
[33]吴洪,卢汉清,马颂德.基于内容图像检索中相关反馈技术的回顾[J].计算机学报,2005,12(10)：25-27.
[34]邸东泉,萧宝瑾.基于移动中心点图像检索的相关反馈技术[J].中国新技术新产品,2009,16(3)：9.
[35]I.Bartolini, P.Ciaccia, M.Patella. WARP:Accurate Retrieval of Shapes Using Phase of Fourier Descriptors and Time Warping Distance[J]. IEEE Transaction on Pattern Analysis and Machine Intelligence,2005,27(3):142-147.
[36]David A.White, Ramesh Jain. Similarity Indexing with the SS-tree[R]. New Orleans:In Proceedings of IEEE International Conference on Data Engineering,1996.
[37]Baback Moghaddam, Qi.Tian, Thomas S.Huang. Spatial Visualization for Content-Based Image Retrieval[R]. Tokyo:In Proceedings of IEEE International Conference on Multimedia and Expo,2001.
[38]张磊,林福宗,张跋.基于支持向量机的相关反馈图像检索算法[J].清华大学学报(自然科学版),2003,42(1)：58-63.
[39]Feng Jing, Li MJ. An Effective Region-Based Image Retrieval Framework[J]. IEEE Transactions on Image Processing,2004,13(5):699-709.
[40]He DA, Nick Cercone, Gu ZM. Applying the Extended Mass-Constraint EM Algorighm to Image Retrieval[J]. Computers and Mathematics with Applications,2008,56(4): 1010-1024.
[41]Han JW, Ngan King N., Li MJ, et al. A Memory Learning Framework for Effective Image Retrieval[J]. IEEE Trans. Image Processing,2005,14(4):511-524.
[42]Yong Rui, Thomas S.Huang, Mehrotra S. Relevance Feedback:A Powerful Tool in Interactive Content-based Image Retrieval[J]. IEEE Transactions on Image Processing,1998, 28(5):644-655.
[43]王济川,郭志刚.LOGISTIC回归模型方法与应用[M].北京：高等教育出版社,2001：14-18.
[44]James O.Ramsay, B.W.Silverman. Functional Data Analysis[M]. New York:Springer Series in Statistics,1997.
[45]Claudia C. On selecting parametric link transformation families in generalized linear models[J]. Journal of Statistical Planning and Inference,1997,61(6):125-139.
[46]陈希孺.广义线性模型(一)[J].数理统计与管理,2002,21(5)：54-61.
[47]丁洁丽,陈希孺.广义线性模型极大似然估计的大样本理论[D].武汉大学：武汉大学概率论与数理统计,2006.
[48]王济川,谢海义,姜宝法.多层统计分析模型[M].北京：高等教育出版社,2008.
[49]茆诗松.贝叶斯统计[M].北京：中国统计出版社,1999：35-48.
[50]朱慧明,韩玉启.贝叶斯多元统计推断理论[M].北京：科学出版社,2006：41-45.
[51]Ingemar J.Cox, Matt L.Miller, Stephen M., et al. Bayesian Relevance Feedback for Image Retrieval[R]. Austria:International Conference on Pattern Recognition,1996.
[52]Su Zhong, Zhang HJ, Stan Li, et al. Relevance Feedback in Content-based Image Retrieval:Bayesian Framework,Feature Subspaces,and Progressive learning[J]. IEEE Transaction on Image Processing,2003,12(8):924-937.
[53]Nuno Vasconcelos, Andrew Lippman. Bayesian Relevance Feedback for Content-based Image Retrieval[R]. Washiongton:In Proceedings of IEEE Workshop on Content-based Access to Image and Video Library,2000.
[54]Nastar C, Meilhac C. Relevance Feedback and Category Search in Image Databases[R]. Florence:International Conference on Multimedia Computing and Systems,1999.
[55]赵玉凤,赵耀,朱振峰.贝叶斯框架下基于区域的相关反馈算法[J].电子与信息学报,2008,30(4)：937-940.