基于概念格的图像语义自动标注
|
摘要
图像语义自动标注是当前图像领域的热点研究问题之一，基于BOV模型的图像表示是典型的图像语义自动标注方法之一，已得到广泛的应用。由于大多数基于BOV模型的图像语义标注关注的是如何提取更恰当视觉单词，忽略了视觉单词和图像语义之间的关系，因而图像语义标注效果较差。概念格是一种有效的数据分析工具，具有表示知识的直观性、完备性和概念的层次性等特点，在许多领域取得了较好的应用效果。本文采用概念格作为语义标注工具，对图像语义自动标注方法进行了研究，其主要研究成果为：
1给出了一种使用概念格约简分析视觉单词的方法。首先用BOV图像表示方法，提取图像中的特征点并量化形成视觉单词；然后把图像标示作为形式背景对象集，图像对应的视觉单词作为形式背景属性集，视觉单词归一化（0-1归一化）结果作为对象和属性之间的关系形成形式背景，采用概念格属性约简的方法求取最小的属性集，以达到约简视觉单词，提高图像语义标注精度的目的；最后实例分析表明，该方法可以有效约简视觉单词。
2给出了一种基于概念格的图像语义标注模型。利用约简后视觉单词作为形式背景，构造其相应的概念格，依据概念格节点外延数确定内涵的重要程度，进而确定视觉单词的重要程度，并使用这些视觉单词标注图像的语义。实验结果分析表明，该方法可以有效地进行图像语义自动标注。
3）在上述研究的基础上，使用MATLAB、JAVA作为开发工具，设计与实现了基于概念格的图像语义标注原型系统，并采用Oliva和Torralba提出的图像库，实验验证了该系统实现图像语义自动标注是可行的、有价值的。
Image semantic annotation is one of the most hot research issue in the fieldof image, and the BOV method is a typical image semantic annotation method，which has been applied widely. Most of the methods of image semanticannotation focus on how to extract more appropriate visual words, but ignore therelationship between the words and images. Concept lattice which is an effectivetool for data analysis, has the feature of intuition, completeness and concepthierarchy on knowledge representation, and has achieved good application effectin many fields. In the thesis, the method of image semantic automatic annotationis studied by using the concept lattice as semantic annotation tool. The mainresearch work is shown as follows:
1A visual words reduction method is presented by using concept lattice asreduction tool. Firstly, feature points in images are extracted, and quantified intovisual words by using BOV image representation. Secondly, the visual words aretransformed into formal context among which images are used as object set,visual words as attribute set, and visual words are normalized as relationship ofobject and attribute. Thirdly the visual words are reduced through attributereduction to improve the precision of image semantic annotation. Instanceanalysis shows that the method can effectively simplify visual words.
2A new model of image semantic annotation is presented based on conceptlattice. By using of the reduced form of concept lattice, the method firstlygenerates corresponding concept lattice structure. Then the important degree ofthe visual words is determined by the quantity of the extension. Finally thoseimportant words are used for image semantic annotation. Experimental resultsshow that the method can effectively carry out the image semantic annotation.
3On the basis of the above research, the concept lattice based imagesemantic annotation prototype system is designed and implemented by usingMATLAB and JAVA as development tools, Oliva and Torralba ‘s image libraryas experimental data. The experiments show that the prototype system is feasibleand valuable.
1给出了一种使用概念格约简分析视觉单词的方法。首先用BOV图像表示方法，提取图像中的特征点并量化形成视觉单词；然后把图像标示作为形式背景对象集，图像对应的视觉单词作为形式背景属性集，视觉单词归一化（0-1归一化）结果作为对象和属性之间的关系形成形式背景，采用概念格属性约简的方法求取最小的属性集，以达到约简视觉单词，提高图像语义标注精度的目的；最后实例分析表明，该方法可以有效约简视觉单词。
2给出了一种基于概念格的图像语义标注模型。利用约简后视觉单词作为形式背景，构造其相应的概念格，依据概念格节点外延数确定内涵的重要程度，进而确定视觉单词的重要程度，并使用这些视觉单词标注图像的语义。实验结果分析表明，该方法可以有效地进行图像语义自动标注。
3）在上述研究的基础上，使用MATLAB、JAVA作为开发工具，设计与实现了基于概念格的图像语义标注原型系统，并采用Oliva和Torralba提出的图像库，实验验证了该系统实现图像语义自动标注是可行的、有价值的。
Image semantic annotation is one of the most hot research issue in the fieldof image, and the BOV method is a typical image semantic annotation method，which has been applied widely. Most of the methods of image semanticannotation focus on how to extract more appropriate visual words, but ignore therelationship between the words and images. Concept lattice which is an effectivetool for data analysis, has the feature of intuition, completeness and concepthierarchy on knowledge representation, and has achieved good application effectin many fields. In the thesis, the method of image semantic automatic annotationis studied by using the concept lattice as semantic annotation tool. The mainresearch work is shown as follows:
1A visual words reduction method is presented by using concept lattice asreduction tool. Firstly, feature points in images are extracted, and quantified intovisual words by using BOV image representation. Secondly, the visual words aretransformed into formal context among which images are used as object set,visual words as attribute set, and visual words are normalized as relationship ofobject and attribute. Thirdly the visual words are reduced through attributereduction to improve the precision of image semantic annotation. Instanceanalysis shows that the method can effectively simplify visual words.
2A new model of image semantic annotation is presented based on conceptlattice. By using of the reduced form of concept lattice, the method firstlygenerates corresponding concept lattice structure. Then the important degree ofthe visual words is determined by the quantity of the extension. Finally thoseimportant words are used for image semantic annotation. Experimental resultsshow that the method can effectively carry out the image semantic annotation.
3On the basis of the above research, the concept lattice based imagesemantic annotation prototype system is designed and implemented by usingMATLAB and JAVA as development tools, Oliva and Torralba ‘s image libraryas experimental data. The experiments show that the prototype system is feasibleand valuable.
引文
[1]解文杰.基于中层语义表示的图像场景分类研究.北京交通大学博士论文,计算机科学与技术专业,2011.
[2]王梅.基于多标签学习的图像语义自动标注研究.复旦大学博士论文,2008.
[3]张素兰,郭平,张继福,胡立华.图像语义自动标注及其粒度分析方法.自动化学报,2012,385:688-697.
[4] Yavlinsky A, Schofield E, Ruger S. Automated image annotation using global featuresand robust nonparametric density estimation. In: Proceedings of the4th InternationalConference on Image and Video Retrieval. Singapore: Springer,2005.PP:507517.
[5] Feng S L, Manmatha R, Lavrenko V. Multiple bemoulli relevance models for image andvideo annotation.In:Proceedings of2004IEEE International Conference on ComputerVision and Pattern Recognition. Washington DC,USA: IEEE,2004. PP:10021009
[6] Bi J, Chen Y, Wang J Z. A sparse support vector machine approach to region-based imagecategorization. In: Proceedings of2005IEEE International Conference on ComputerVision and Pattern Recognition. San Diego, California:IEEE,2005.PP:11211128
[7] Zhang Q, Goldman S A, Yu W, Fritts J. Content-based image retrieval usingmultiple-instance learning. In: Proceedings of the19th International Conference onMachine Leaning. San Fransisco, USA:Morgan Kaufinann,2002.,PP:682689
[8]谢昭等.图像理解理论与方法.北京:科学出版社,2009.
[9] Swain M J, Ballard D H. Color indexing. International Journal of Computer Vision,1991,7(1):1132.
[10]梁柱.基于视觉单词树的快速图像检索.西南师范大学学报自然科学版,2010,353，235-238.
[11]王梅.基于多标签学习的图像语义自动标注研究.复旦大学信息科学与工程学院博士论文,2008.
[12] Godin R. Incremental concept formation algorithm based on Galois (concept) lattices[J].Computational Intelligence,1995,11(2):246-267.
[13] Capineto C, Romano G. Galois: an order-theoretic approach to conceptual clustering.Proceedings of10th International Conference on Machine Learning, Amherst. June1993,PP:33-40.
[14] Ho T B. An approach to concept formation based on formal concept analysis[J].IEICETrans. Information and Systems,1995,E78-D(5):553-559
[15] Nourine L, Raynaud O. A fast algorithm for building lattices [A]. Workshop onComputational Graph Theory and Combinatorics [C]. Victoria, Canada, May,1999.
[16]王浩,胡学钢,赵文兵.基于量化相对约简格的分类规则发现.复旦学报(自然科学版),2004,435:761-764.
[17]胡可云,陆玉昌,石纯一.基于概念格的分类和关联规则的集成挖掘方法.软件学报,2000,11(11):1478-1484.
[18]王志海,胡可云,胡学钢,刘宗田,张奠成.概念格上规则提取的一般算法与渐进式算法.计算机学报,1999,221:66-70.
[19]谢志鹏,刘宗田.基于概念格的关联规则发现.小型微型计算机系统,2000,2110:1028-1031.
[20] Ganter B,Wille.R著,马垣,张学东,迟呈英,王丽君,等译.形式概念分析.科学出版社,2007.
[21]胡立华,张继福,张素兰.基于剪枝的概念格渐进式构造.计算机应用,2006,267:1659-1661.
[22]张文修,魏玲,祁建军.概念格的属性约简理论与方法.中国科学E信息科学,2005,35(6):628-639.
[23] Shao Ming wen. The Reduction for Two Kind of Generalized Concept Lattice//Proc ofthe4th International Conference on Machine Learning and Cybernetics. Guangzhou,China,2005, PP:2217-2222.
[24]林培榕,张其森,李进金.基于交可约等价类的概念格属性约简.模式识别与人工智能,2010,235:720-725.
[25] Yao Yi yu,ChenYao hua.Rough Set Approximations in Formal Concept Analysis//Deters J F,Skowron A,ed s. Trans on Rough Set V. Berlin, Germany: Springer,2006,PP:285-305.
[26]李立峰,王国俊.一种求概念格属性约简的方法.计算机工程与应用,2006,42(20):147-149.
[27]吕跃进,李金海.基于一一映射的概念格属性约简算法.计算机应用研究,2009,26(3)：849-851.
[28]刘硕研,须德,冯松鹤,刘镝,裘正定.一种基于上下文语义信息的图像块视觉单词生成算法.电子学报,2010,385:1156-1161.
[29]江悦,王润生.基于多特征扩展PLSA模型的场景图像分类.信号处理,2010,264539-544
[30]李志欣,施智平,李志清,史忠植.融合语义主题的图像自动标注.软件学报,2011,22(4):801812.
[31]王宇石,高文.用基于视觉单词上下文的核函数对图像分类.中国图像图形学报,2010,154:607-616.
[32] Lowe D G. Object Recognition from Loca1Scale2Invariant Features[C]//Proceedingsof the7th International Conference on Computer Vision,1999,PP:1150-1157.
[33] Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints [J]. InternationalJournal of Computer Vision,2004,60(2):91-110.
[34] Lindeberg T. Scale space theory: A basic tool for analyzing structures at different scales[J]. Journal of Applied Statistics,1994,212:224-270.
[35] Brown，M and David G. Lowe. Invariant features from interest point groups. In Proc. OfBritish Machine Vision Conference,2002, PP:656-665.
[2]王梅.基于多标签学习的图像语义自动标注研究.复旦大学博士论文,2008.
[3]张素兰,郭平,张继福,胡立华.图像语义自动标注及其粒度分析方法.自动化学报,2012,385:688-697.
[4] Yavlinsky A, Schofield E, Ruger S. Automated image annotation using global featuresand robust nonparametric density estimation. In: Proceedings of the4th InternationalConference on Image and Video Retrieval. Singapore: Springer,2005.PP:507517.
[5] Feng S L, Manmatha R, Lavrenko V. Multiple bemoulli relevance models for image andvideo annotation.In:Proceedings of2004IEEE International Conference on ComputerVision and Pattern Recognition. Washington DC,USA: IEEE,2004. PP:10021009
[6] Bi J, Chen Y, Wang J Z. A sparse support vector machine approach to region-based imagecategorization. In: Proceedings of2005IEEE International Conference on ComputerVision and Pattern Recognition. San Diego, California:IEEE,2005.PP:11211128
[7] Zhang Q, Goldman S A, Yu W, Fritts J. Content-based image retrieval usingmultiple-instance learning. In: Proceedings of the19th International Conference onMachine Leaning. San Fransisco, USA:Morgan Kaufinann,2002.,PP:682689
[8]谢昭等.图像理解理论与方法.北京:科学出版社,2009.
[9] Swain M J, Ballard D H. Color indexing. International Journal of Computer Vision,1991,7(1):1132.
[10]梁柱.基于视觉单词树的快速图像检索.西南师范大学学报自然科学版,2010,353，235-238.
[11]王梅.基于多标签学习的图像语义自动标注研究.复旦大学信息科学与工程学院博士论文,2008.
[12] Godin R. Incremental concept formation algorithm based on Galois (concept) lattices[J].Computational Intelligence,1995,11(2):246-267.
[13] Capineto C, Romano G. Galois: an order-theoretic approach to conceptual clustering.Proceedings of10th International Conference on Machine Learning, Amherst. June1993,PP:33-40.
[14] Ho T B. An approach to concept formation based on formal concept analysis[J].IEICETrans. Information and Systems,1995,E78-D(5):553-559
[15] Nourine L, Raynaud O. A fast algorithm for building lattices [A]. Workshop onComputational Graph Theory and Combinatorics [C]. Victoria, Canada, May,1999.
[16]王浩,胡学钢,赵文兵.基于量化相对约简格的分类规则发现.复旦学报(自然科学版),2004,435:761-764.
[17]胡可云,陆玉昌,石纯一.基于概念格的分类和关联规则的集成挖掘方法.软件学报,2000,11(11):1478-1484.
[18]王志海,胡可云,胡学钢,刘宗田,张奠成.概念格上规则提取的一般算法与渐进式算法.计算机学报,1999,221:66-70.
[19]谢志鹏,刘宗田.基于概念格的关联规则发现.小型微型计算机系统,2000,2110:1028-1031.
[20] Ganter B,Wille.R著,马垣,张学东,迟呈英,王丽君,等译.形式概念分析.科学出版社,2007.
[21]胡立华,张继福,张素兰.基于剪枝的概念格渐进式构造.计算机应用,2006,267:1659-1661.
[22]张文修,魏玲,祁建军.概念格的属性约简理论与方法.中国科学E信息科学,2005,35(6):628-639.
[23] Shao Ming wen. The Reduction for Two Kind of Generalized Concept Lattice//Proc ofthe4th International Conference on Machine Learning and Cybernetics. Guangzhou,China,2005, PP:2217-2222.
[24]林培榕,张其森,李进金.基于交可约等价类的概念格属性约简.模式识别与人工智能,2010,235:720-725.
[25] Yao Yi yu,ChenYao hua.Rough Set Approximations in Formal Concept Analysis//Deters J F,Skowron A,ed s. Trans on Rough Set V. Berlin, Germany: Springer,2006,PP:285-305.
[26]李立峰,王国俊.一种求概念格属性约简的方法.计算机工程与应用,2006,42(20):147-149.
[27]吕跃进,李金海.基于一一映射的概念格属性约简算法.计算机应用研究,2009,26(3)：849-851.
[28]刘硕研,须德,冯松鹤,刘镝,裘正定.一种基于上下文语义信息的图像块视觉单词生成算法.电子学报,2010,385:1156-1161.
[29]江悦,王润生.基于多特征扩展PLSA模型的场景图像分类.信号处理,2010,264539-544
[30]李志欣,施智平,李志清,史忠植.融合语义主题的图像自动标注.软件学报,2011,22(4):801812.
[31]王宇石,高文.用基于视觉单词上下文的核函数对图像分类.中国图像图形学报,2010,154:607-616.
[32] Lowe D G. Object Recognition from Loca1Scale2Invariant Features[C]//Proceedingsof the7th International Conference on Computer Vision,1999,PP:1150-1157.
[33] Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints [J]. InternationalJournal of Computer Vision,2004,60(2):91-110.
[34] Lindeberg T. Scale space theory: A basic tool for analyzing structures at different scales[J]. Journal of Applied Statistics,1994,212:224-270.
[35] Brown，M and David G. Lowe. Invariant features from interest point groups. In Proc. OfBritish Machine Vision Conference,2002, PP:656-665.