基于混合属性的零样本图像分类
|
摘要
对于具有相似属性的类别而言,在有限维度的语义属性下,基于属性的零样本图像分类器难以对它们进行正确区分.考虑到语义属性描述类别的有限性,在直接属性预测(Direct Attribute Prediction,DAP)模型的基础上,提出一种基于混合属性的零样本图像分类模型(Hybrid Attribute-Based DAP,HA-DAP).首先,对样本的底层特征进行稀疏编码并利用编码后的非语义属性来辅助现有的语义属性;将非语义属性与语义属性构成混合属性并将其作为DAP模型的属性中间层,利用属性预测模型的思想进行混合属性分类器的训练;最后,根据预测的混合属性以及属性与类别之间的关系进行测试样本类别标签的预测.在OSR、Pub Fig以及Shoes数据集上的实验结果表明,HA-DAP的分类性能优于DAP,不仅能够取得较高的零样本图像分类精度,而且还获得了较高的AUC值.
When the dimensionality of the semantic attributes is limited,it is difficult for attribute-based zero-shot image classifiers to distinguish the objects with similar attributes. Aiming at the limitation of describing objects with semantic attributes,an improved direct attribute prediction( DAP) model for zero-shot image classifying based on hybrid attribute( HA) is proposed,which is called HA-DAP. At first,we carry out the sparse coding on the low-level features to obtain the non-semantic attributes that are used to assist the existing semantic attributes. Then,we take the hybrid attributes including the learned non-semantic attributes and the manually specified semantic attributes as the mid-layer of DAP model and use the idea of attribute prediction to train the hybrid attribute-based classifier. At last,according to the predicted hybrid attributes and the relationship between the attributes and classes,we can recognize the class label for the testing sample. Experimental results on the OSR,Pub Fig and Shoes datasets show that,the HA-DAP outperforms the DAP in the classification performance,i. e.,when compared with the DAP,the proposed HA-DAP yields much higher zero-shot image classification accuracy and AUC value.
When the dimensionality of the semantic attributes is limited,it is difficult for attribute-based zero-shot image classifiers to distinguish the objects with similar attributes. Aiming at the limitation of describing objects with semantic attributes,an improved direct attribute prediction( DAP) model for zero-shot image classifying based on hybrid attribute( HA) is proposed,which is called HA-DAP. At first,we carry out the sparse coding on the low-level features to obtain the non-semantic attributes that are used to assist the existing semantic attributes. Then,we take the hybrid attributes including the learned non-semantic attributes and the manually specified semantic attributes as the mid-layer of DAP model and use the idea of attribute prediction to train the hybrid attribute-based classifier. At last,according to the predicted hybrid attributes and the relationship between the attributes and classes,we can recognize the class label for the testing sample. Experimental results on the OSR,Pub Fig and Shoes datasets show that,the HA-DAP outperforms the DAP in the classification performance,i. e.,when compared with the DAP,the proposed HA-DAP yields much higher zero-shot image classification accuracy and AUC value.
引文
[1]Lampert C H,Nickisch H,Harmeling S.Attribute-based classification for zero-shot visual object categorization[J].Pattern Analysis and Machine Intelligence,2014,36(3):453-465.
[2]Douze M,Ramisa A,Schmid C.Combining attributes and Fisher vectors for efficient image retrieval[A].Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition[C].Colorado Springs:IEEE Press,2011.745-752.
[3]B Siddiquie,R Feris,L Davis.Image ranking and retrieval based on multi-attribute queries[A].Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition[C].Colorado Springs:IEEE Press,2011.801-808.
[4]Farhadi A,Endres I,Hoiem D,et al.Describing objects by their attributes[A].Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition[C].Miami:IEEE Press,2009.1778-1785.
[5]Kumar N,Berg A,Belhumeur P,et al.Describable visual attributes for face verification and image search[J].Pattern Analysis and Machine Intelligence,2011,33(10):1962-1977.
[6]Mahajan D,Sellamanickam S,Nair V.A joint learning framework for attribute models and object descriptions[A].Proceedings of the IEEE International Conference on Computer Vision[C].Barcelona:IEEE Press,2011.1227-1234.
[7]Liu M X,Zhang D Q,Chen S C.Attribute relation learning for zero-shot classification[J].Neurocomputing,2014,139(2):34-46.
[8]Larochelle H,Erhan E,Bengio Y.Zero-data learning of new tasks[A].Proceedings of the International Conference on Artificial Intelligence[C].Chicago:AAAI Press,2008.646-651.
[9]Yu X,Aloimonos Y.Attribute-based transfer learning for object categorization with zero/one training example[A].Proceedings of the International Conference on Computer Vision[C].Heraklion:Springer Verlag Press,2010.127-140.
[10]Lampert C H,Nickisch H,Harmeling S.Learning to detect unseen object classes by between-class attribute transfer[A].Proceedings of the IEEE International Computer Vision and Pattern Recognition[C].Miami:IEEE Press,2009.951-958.
[11]石光明,刘丹华,高大化,等.压缩感知理论及其研究进展[J].电子学报,2009,37(5):1071-1078.Shi Guang-ming,Liu Dan-hua,Gao Da-hua,et al.Advances in theory and application of compressed sensing[J].Acta Electronica Sinica,2009,37(5):1071-1078.(in Chinese)
[12]Sharmanska V,Quadrianto N,Lampert C H.Augmented attribute representations[A].Proceedings of the International Conference on Computer Vision[C].Florence:Springer Verlag Press,2012.242-255.
[13]亓晓振,王庆.一种基于稀疏编码的多核学习图像分类算法[J].电子学报,2012,40(4):773-779.Qi Xiao-zhen,Wang Qing.An image classification approach based on sparse coding and multiple kernel learning[J].Acta Electronica Sinica,2012,40(4):773-779.(in Chinese)
[14]黄丽丽,肖亮,韦至辉.彩色图像去马赛克的非局部稀疏表示方法[J].电子学报,2014,42(2):272-279.Huang Li-li,Xiao Liang,Wei Zhi-hui.A nonlocal sparse representation method for color demosaicking[J].Acta Electronica Sinica,2014,42(2):272-279.(in Chinese)
[15]孙玉宝,韦志辉,肖亮,等.多形态稀疏性正则化的图像超分辨率算法[J].电子学报,2010,38(12):2898-2903.Sun Yu-bao,Wei Zhi-hui,Xiao Liang,et al.Multimorphology sparsity regularized image super-resolution[J].Acta Electronica Sinica,2010,38(12):2898-2903.(in Chinese)
[16]Fawcett Tom.An introduction to ROC analysis[J].Pattern Recognition Letters,2006,27(8):861-874.
[2]Douze M,Ramisa A,Schmid C.Combining attributes and Fisher vectors for efficient image retrieval[A].Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition[C].Colorado Springs:IEEE Press,2011.745-752.
[3]B Siddiquie,R Feris,L Davis.Image ranking and retrieval based on multi-attribute queries[A].Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition[C].Colorado Springs:IEEE Press,2011.801-808.
[4]Farhadi A,Endres I,Hoiem D,et al.Describing objects by their attributes[A].Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition[C].Miami:IEEE Press,2009.1778-1785.
[5]Kumar N,Berg A,Belhumeur P,et al.Describable visual attributes for face verification and image search[J].Pattern Analysis and Machine Intelligence,2011,33(10):1962-1977.
[6]Mahajan D,Sellamanickam S,Nair V.A joint learning framework for attribute models and object descriptions[A].Proceedings of the IEEE International Conference on Computer Vision[C].Barcelona:IEEE Press,2011.1227-1234.
[7]Liu M X,Zhang D Q,Chen S C.Attribute relation learning for zero-shot classification[J].Neurocomputing,2014,139(2):34-46.
[8]Larochelle H,Erhan E,Bengio Y.Zero-data learning of new tasks[A].Proceedings of the International Conference on Artificial Intelligence[C].Chicago:AAAI Press,2008.646-651.
[9]Yu X,Aloimonos Y.Attribute-based transfer learning for object categorization with zero/one training example[A].Proceedings of the International Conference on Computer Vision[C].Heraklion:Springer Verlag Press,2010.127-140.
[10]Lampert C H,Nickisch H,Harmeling S.Learning to detect unseen object classes by between-class attribute transfer[A].Proceedings of the IEEE International Computer Vision and Pattern Recognition[C].Miami:IEEE Press,2009.951-958.
[11]石光明,刘丹华,高大化,等.压缩感知理论及其研究进展[J].电子学报,2009,37(5):1071-1078.Shi Guang-ming,Liu Dan-hua,Gao Da-hua,et al.Advances in theory and application of compressed sensing[J].Acta Electronica Sinica,2009,37(5):1071-1078.(in Chinese)
[12]Sharmanska V,Quadrianto N,Lampert C H.Augmented attribute representations[A].Proceedings of the International Conference on Computer Vision[C].Florence:Springer Verlag Press,2012.242-255.
[13]亓晓振,王庆.一种基于稀疏编码的多核学习图像分类算法[J].电子学报,2012,40(4):773-779.Qi Xiao-zhen,Wang Qing.An image classification approach based on sparse coding and multiple kernel learning[J].Acta Electronica Sinica,2012,40(4):773-779.(in Chinese)
[14]黄丽丽,肖亮,韦至辉.彩色图像去马赛克的非局部稀疏表示方法[J].电子学报,2014,42(2):272-279.Huang Li-li,Xiao Liang,Wei Zhi-hui.A nonlocal sparse representation method for color demosaicking[J].Acta Electronica Sinica,2014,42(2):272-279.(in Chinese)
[15]孙玉宝,韦志辉,肖亮,等.多形态稀疏性正则化的图像超分辨率算法[J].电子学报,2010,38(12):2898-2903.Sun Yu-bao,Wei Zhi-hui,Xiao Liang,et al.Multimorphology sparsity regularized image super-resolution[J].Acta Electronica Sinica,2010,38(12):2898-2903.(in Chinese)
[16]Fawcett Tom.An introduction to ROC analysis[J].Pattern Recognition Letters,2006,27(8):861-874.