基于主题融合和关联规则挖掘的图像标注
|
摘要
为减小"语义鸿沟",在LDA主题模型的基础上,提出了一种主题融合和关联规则挖掘的图像标注方法。首先,针对视觉和文本信息的关联度不高的问题,引入基于向量机的多类别分类得到图像的类别信息。其次,通过文本模态的语义主题分布和类别信息,计算出图像类的文本主题分布。未知图像将其所属类的文本主题分布与其视觉主题分布进行加权融合,并以此概率模型计算初始标签集。最后依据初始标注词概率,利用关联规则挖掘和词间相关性挖掘文本关联度,从而得到精确化语义标注。在Corel5K图像数据集上进行对比实验,实验结果证明了方法的有效性。
In order to reduce the "semantic gap",based on the LDA topic model,an image annotation approach which uses topics fusion and association rule mining was proposed.First,to solve the problem of low correlation between visualand text information,the vector machine-based multi-category classification is introduced to obtain the category information of the image.Then,the text topic distribution of the image class is calculated by the semantic topic distribution and classification information of the text modality.The unknown image weights the text topic distribution of its class and its visual topic distribution,and calculates the initial label set using this probability model.Finally,based on the probability of initial label words,the association rules mining and inter-word correlation are used to mine the text relevance to obtain precise semantic annotation.The comparative experiments were carried out on the Corel5 K image dataset.The experimental results show the effectiveness of the proposed method.
In order to reduce the "semantic gap",based on the LDA topic model,an image annotation approach which uses topics fusion and association rule mining was proposed.First,to solve the problem of low correlation between visualand text information,the vector machine-based multi-category classification is introduced to obtain the category information of the image.Then,the text topic distribution of the image class is calculated by the semantic topic distribution and classification information of the text modality.The unknown image weights the text topic distribution of its class and its visual topic distribution,and calculates the initial label set using this probability model.Finally,based on the probability of initial label words,the association rules mining and inter-word correlation are used to mine the text relevance to obtain precise semantic annotation.The comparative experiments were carried out on the Corel5 K image dataset.The experimental results show the effectiveness of the proposed method.
引文
[1] GU Y,XUE H Y,YANG J.Cross-modal saliency correlation for image annotation[J].Neural Processing Letters,2017,45(3):777-789.
[2] HE C,CHEN Z X,LIU C Y.Bottom-up image saliency target detection via bottomup[J].Journal of Optoelectronic·Laser,2016,27(2):886-892.(in Chinese)贺超,陈振学,刘学云.自底向上的图像显著目标检测研究[J].光电子·激光,2016,27(2):886-892.
[3] DUYGULU P,BARNARD K,DE FREITAS J,et al.Object re- cognition as machine translation:Learning a lexicon for a fixed image vocabulary[C]//Proceedings of the 7th European Confe-rence of Computer Vision.Copenhagen,Kongeriget,Danmark:ECCV,2002:97-112.
[4] JEON J,LAVRENKO V,MANMATHA R.Automatic image annotation and retrieval using cross-media relevance models[C]∥Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.2003:119-126.
[5] LAVRENKO V,MANMATHA R,JEON J.A model for lear- ning the semantics of pictures[C]//Adwances in Neural Information Processing Systems 16.Cambridge:MIT Press,2004:553-560.
[6] FENG S L,MANMATHA R,LAVRENKO V.Multiple Ber- noulli relevance models for image and video annotation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington:IEEE Computer Society,2004:1002-1009.
[7] MORALES-GONZALEZ A,GARCIA-REYES E,SUCAR L E.Unsupervised segmentation evaluation for image annotation[C]//10th International Conference on Computer Vision Theory and Application.Berlin,Germany,2015.
[8] HOFMANN T.Unsupervised learning by probabilistic latent semantic analysis[J].Machine Learning,2001,42(1-2):177-196.
[9] BLEI D M,NG A Y,JORDAN M I.Latent Dirichlet allocation[J].Journal of Machine Learning Research,2003,3:993-1022.
[10] MONAY F,GATICA-PEREZ D.Modeling semantic aspects for cross-media image indexing [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(10):1802-1817.
[11] LI Z X,SHI Z P,LI Z Q,et al.Automatic Image Annotation by Fusing Semantic topics[J].Journal of Software,2011,22(4):801-812.(in Chinese)李志欣,施志平,李志清,等.融合语义主题的图像自动标注[J].软件学报,2011,22(4):801-812.
[12] LI Z X,SHI Z P,ZHANG C L,et al.Hybrid generative/discri- minative model for automatic image annotation[J].Journal of Ima-ge and Graphics,2015,20(5):687-699.(in Chinese)李志欣,施志平,张灿龙,等.混合生成式和判别式模型的图像自动标注[J].中国图象图形学报,2015,20(5):687-699.
[13] ZHAO P,WANG W B,ZHU W W,et al.Automatic image annotation by combining aspects and visual semantics[J].Journal of Computer Aided Design & Computer Graphics,2013,25(11):1709-1714.(in Chinese)赵鹏,王文彬,朱伟伟,等.融合主题和视觉语义的图像自动标注方法[J].计算机辅助设计与图形学学报,2013,25(11):1709-1714.
[14] SUN J D,LI H H,JIN J L,et al.Image automatic annotation based on visual semantic topics[J].Measurement & Control Technology,2016,35(12):11-15.(in Chinese)孙君顶,李海华,靳姣林,等.基于视觉语义主题的图像自动标注[J].测控技术,2016,35(12):11-15.
[15] SUN J D,LI H H,JIN J L,et al.Image Automatic annotation based on multi-feature fusion and PLSA-GMM[J].Measurement & Control Technology,2017,36(4):31-35.(in Chinese)孙君顶,李海华,靳姣林,等.基于多特征融合与PLSA-GMM的图像自动标注[J].测控技术,2017,36(4):31-35.
[16] SUN J D,LI H H,JIN J L.Image automatic annotation based on the visual semantic topics and feedback log[J].Journal of Optoelectronics.Laser,2017,28(4):441-450.(in Chinese)孙君顶,李海华,靳姣林,等.基于视觉语义主题与反馈日志的图像自动标注[J].光电子·激光,2017,28(4):441-450.
[17] TIAN D P.Integrating PLSA and random walk model for automatic image annotation[J].Journal of Chinese Computer Systems,2017,38(8):1899-1907.(in Chinese)田东平.融合PLSA和随机游走模型的自动图像标注[J].小型微型计算机系统,2017,38(8):1899-1907.
[18] CAO J,LOU J X,LI X X.Image annotation probabilistic topic model improving corr-LDA model[J].Journal of Chinese Computer Systems,2017,38(3):615-619.
[19] CAO J,LOU J X,LI X X.Image annotation probabilistic topic model fusing class information[J].Computer Engineering and Applications,2017,53(10):187-192.
[20] SUN D D,GE M L,DING Z L,et al.Tagging enrichment algorithm based on tag semantic and image visual[J].Journal of Chinese Computer Systems.2017,38(4):886-890.(in Chinese)孙登第,葛美玲,丁转莲,等.基于标注词语义与图像视觉的标签丰富算法[J].小型微型计算机系统,2017,38(4):886-890.
[21] CHANG C C,LIN C J.LIBSVM:A library for support vector machines[J].ACM Transactions on Intelligent Systems and Technology(TIST),2011,2(3):27.
[22] HAN J,PEI J,YIN Y.Mining frequent patterns without candidate generation[C]//Proceedings of the ACM SIGMOD International Conference on Management of Data.USA,2000:1-12.
[23] GAO Y Y,YIN Y X,UOZUMI T.A hierarchical image annotation method based on SVM and semi-supervised EM [J].Acta Automatica Sinica,2010,36(7):960-967.
[24] ZHOU S J,MENG J,HUANG Z P,et al.A method for discrimination of processed ginger based on image color feature and support vector machine model[J].Analytical Methods,2016,8:2201-2206.
[25] PONTI M,NAZARE T S,THUME G S.Image quantization as a dimensionality reduction procedure in color and texture feature extraction[J].Neurocomputing,2016,173:385-396.
[2] HE C,CHEN Z X,LIU C Y.Bottom-up image saliency target detection via bottomup[J].Journal of Optoelectronic·Laser,2016,27(2):886-892.(in Chinese)贺超,陈振学,刘学云.自底向上的图像显著目标检测研究[J].光电子·激光,2016,27(2):886-892.
[3] DUYGULU P,BARNARD K,DE FREITAS J,et al.Object re- cognition as machine translation:Learning a lexicon for a fixed image vocabulary[C]//Proceedings of the 7th European Confe-rence of Computer Vision.Copenhagen,Kongeriget,Danmark:ECCV,2002:97-112.
[4] JEON J,LAVRENKO V,MANMATHA R.Automatic image annotation and retrieval using cross-media relevance models[C]∥Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.2003:119-126.
[5] LAVRENKO V,MANMATHA R,JEON J.A model for lear- ning the semantics of pictures[C]//Adwances in Neural Information Processing Systems 16.Cambridge:MIT Press,2004:553-560.
[6] FENG S L,MANMATHA R,LAVRENKO V.Multiple Ber- noulli relevance models for image and video annotation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington:IEEE Computer Society,2004:1002-1009.
[7] MORALES-GONZALEZ A,GARCIA-REYES E,SUCAR L E.Unsupervised segmentation evaluation for image annotation[C]//10th International Conference on Computer Vision Theory and Application.Berlin,Germany,2015.
[8] HOFMANN T.Unsupervised learning by probabilistic latent semantic analysis[J].Machine Learning,2001,42(1-2):177-196.
[9] BLEI D M,NG A Y,JORDAN M I.Latent Dirichlet allocation[J].Journal of Machine Learning Research,2003,3:993-1022.
[10] MONAY F,GATICA-PEREZ D.Modeling semantic aspects for cross-media image indexing [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(10):1802-1817.
[11] LI Z X,SHI Z P,LI Z Q,et al.Automatic Image Annotation by Fusing Semantic topics[J].Journal of Software,2011,22(4):801-812.(in Chinese)李志欣,施志平,李志清,等.融合语义主题的图像自动标注[J].软件学报,2011,22(4):801-812.
[12] LI Z X,SHI Z P,ZHANG C L,et al.Hybrid generative/discri- minative model for automatic image annotation[J].Journal of Ima-ge and Graphics,2015,20(5):687-699.(in Chinese)李志欣,施志平,张灿龙,等.混合生成式和判别式模型的图像自动标注[J].中国图象图形学报,2015,20(5):687-699.
[13] ZHAO P,WANG W B,ZHU W W,et al.Automatic image annotation by combining aspects and visual semantics[J].Journal of Computer Aided Design & Computer Graphics,2013,25(11):1709-1714.(in Chinese)赵鹏,王文彬,朱伟伟,等.融合主题和视觉语义的图像自动标注方法[J].计算机辅助设计与图形学学报,2013,25(11):1709-1714.
[14] SUN J D,LI H H,JIN J L,et al.Image automatic annotation based on visual semantic topics[J].Measurement & Control Technology,2016,35(12):11-15.(in Chinese)孙君顶,李海华,靳姣林,等.基于视觉语义主题的图像自动标注[J].测控技术,2016,35(12):11-15.
[15] SUN J D,LI H H,JIN J L,et al.Image Automatic annotation based on multi-feature fusion and PLSA-GMM[J].Measurement & Control Technology,2017,36(4):31-35.(in Chinese)孙君顶,李海华,靳姣林,等.基于多特征融合与PLSA-GMM的图像自动标注[J].测控技术,2017,36(4):31-35.
[16] SUN J D,LI H H,JIN J L.Image automatic annotation based on the visual semantic topics and feedback log[J].Journal of Optoelectronics.Laser,2017,28(4):441-450.(in Chinese)孙君顶,李海华,靳姣林,等.基于视觉语义主题与反馈日志的图像自动标注[J].光电子·激光,2017,28(4):441-450.
[17] TIAN D P.Integrating PLSA and random walk model for automatic image annotation[J].Journal of Chinese Computer Systems,2017,38(8):1899-1907.(in Chinese)田东平.融合PLSA和随机游走模型的自动图像标注[J].小型微型计算机系统,2017,38(8):1899-1907.
[18] CAO J,LOU J X,LI X X.Image annotation probabilistic topic model improving corr-LDA model[J].Journal of Chinese Computer Systems,2017,38(3):615-619.
[19] CAO J,LOU J X,LI X X.Image annotation probabilistic topic model fusing class information[J].Computer Engineering and Applications,2017,53(10):187-192.
[20] SUN D D,GE M L,DING Z L,et al.Tagging enrichment algorithm based on tag semantic and image visual[J].Journal of Chinese Computer Systems.2017,38(4):886-890.(in Chinese)孙登第,葛美玲,丁转莲,等.基于标注词语义与图像视觉的标签丰富算法[J].小型微型计算机系统,2017,38(4):886-890.
[21] CHANG C C,LIN C J.LIBSVM:A library for support vector machines[J].ACM Transactions on Intelligent Systems and Technology(TIST),2011,2(3):27.
[22] HAN J,PEI J,YIN Y.Mining frequent patterns without candidate generation[C]//Proceedings of the ACM SIGMOD International Conference on Management of Data.USA,2000:1-12.
[23] GAO Y Y,YIN Y X,UOZUMI T.A hierarchical image annotation method based on SVM and semi-supervised EM [J].Acta Automatica Sinica,2010,36(7):960-967.
[24] ZHOU S J,MENG J,HUANG Z P,et al.A method for discrimination of processed ginger based on image color feature and support vector machine model[J].Analytical Methods,2016,8:2201-2206.
[25] PONTI M,NAZARE T S,THUME G S.Image quantization as a dimensionality reduction procedure in color and texture feature extraction[J].Neurocomputing,2016,173:385-396.