融合萤火虫方法的多标签懒惰学习算法
|
摘要
已有的多标签懒惰学习算法(IMLLA)在利用近邻标签时因仅考虑了近邻标签相关性信息,而忽略相似度的影响,这可能会使算法的鲁棒性有所降低。针对这个问题,引入萤火虫方法,将相似度信息与标签信息相结合,提出一种融合萤火虫方法的多标签懒惰学习算法(FF-MLLA)。首先,利用Minkowski距离来度量样本间相似度,从而找到近邻点;然后,结合标签近邻点和萤火虫方法对标签计数向量进行改进;最后,使用奇异值分解(SVD)与核极限学习机(ELM)进行线性分类。该算法同时考虑了标签信息与相似度信息从而提高了鲁棒性。实验结果表明,所提算法较其他的多标签学习算法有一定优势,并使用统计假设检验与稳定性分析进一步说明所提出算法的合理性与有效性。
The existing Improved Multi-label Lazy Learning Approach(IMLLA) has the problem that the influence of similarity information is ignored with only the neighbor label correlation information considered when the neighbor labels were used, which may reduce the robustness of the approach. To solve this problem, with firefly method introduced and the combination of similarity information with label information, a Multi-label Lazy Learning Approach based on FireFly method(FF-MLLA) was proposed. Firstly, Minkowski distance was used to measure the similarity between samples to find the neighbor point. Secondly, the label count vector was improved by combining the neighbor point and firefly method. Finally, Singular Value Decomposition(SVD) and kernel Extreme Learning Machine(ELM) were used to realize linear classification. The robustness of the approach was improved due to considering both label information and similarity information. The experimental results demonstrate that the proposed approach improves the classification performance to a great extent compared to other multi-label learning approaches. And the statistical hypothesis testing and stability analysis are used to further illustrate the rationality and effectiveness of the proposed approach.
The existing Improved Multi-label Lazy Learning Approach(IMLLA) has the problem that the influence of similarity information is ignored with only the neighbor label correlation information considered when the neighbor labels were used, which may reduce the robustness of the approach. To solve this problem, with firefly method introduced and the combination of similarity information with label information, a Multi-label Lazy Learning Approach based on FireFly method(FF-MLLA) was proposed. Firstly, Minkowski distance was used to measure the similarity between samples to find the neighbor point. Secondly, the label count vector was improved by combining the neighbor point and firefly method. Finally, Singular Value Decomposition(SVD) and kernel Extreme Learning Machine(ELM) were used to realize linear classification. The robustness of the approach was improved due to considering both label information and similarity information. The experimental results demonstrate that the proposed approach improves the classification performance to a great extent compared to other multi-label learning approaches. And the statistical hypothesis testing and stability analysis are used to further illustrate the rationality and effectiveness of the proposed approach.
引文
[1] GIBAJA E,VENTURA S.A tutorial on multilabel learning[J].ACM Computing Surveys,2015,47(3):1-38.
[2] 何志芬,杨明,刘会东.多标记分类和标记相关性的联合学习[J].软件学报,2014,25(9):1967-1981.(HE Z F,YANG M,LIU H D.Joint learning of multi-label classification and label correlations[J].Journal of Software,2014,25(9):1967-1981.)
[3] LIU J,CHANG W,WU Y,et al.Deep learning for extreme multi-label text classification[C]// Proceedings of the 40th International ACM SIGIR Conference on Research & Development in Information Retrieval.New York:ACM,2017:115-124.
[4] KORDMAHALLEH M M,HOMAIFAR A,DUKKA B K C.Hierarchical multi-label gene function prediction using adaptive mutation in crowding niching[C]// Proceedings of the 13th IEEE International Conference on BioInformatics and BioEngineering.Piscataway,NJ:IEEE,2013:1-6.
[5] ZHU X,LI X,ZHANG S.Block-row sparse multiview multilabel learning for image classification[J].IEEE Transactions on Cybernetics,2016,46(2):450.
[6] WANG Z,CHEN T,LI G,et al.Multi-label image recognition by recurrently discovering attentional regions[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision.Washington,DC:IEEE Computer Society,2017:464-472.
[7] OZONAT K M,YOUNG D E.Towards a universal marketplace over the Web:statistical multi-label classification of service provider forms with simulated annealing[C]// Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York:ACM,2009:1295-1304.
[8] HOU S,ZHOU S,CHEN L,et al.Multi-label learning with label relevance in advertising video[J].Neurocomputing,2016,171(C):932-948.
[9] BOUTELL M R,LUO J,SHEN X,et al.Learning multi-label scene classification [J].Pattern Recognition,2004,37(9):1757-1771.
[10] ZHANG M,ZHOU Z.ML-KNN:a lazy learning approach to multi-label learning[J].Pattern Recognition,2007,40(7):2038-2048.
[11] LEE J,KIM H,KIM N R,et al.An approach for multi-label classification by directed acyclic graph with label correlation maximization[J].Information Sciences,2016,351(C):101-114.
[12] ELISSEEFF A E,WESTON J.A kernel method for multi-labelled classification[C]// Proceedings of the 14th International Conference on Neural Information Processing Systems:Natural and Synthetic.Cambridge,MA:MIT Press,2002:681-687.
[13] HUANG G,ZHU Q,SIEW C K.Extreme learning machine:theory and applications[J].Neurocomputing,2006,70(1/2/3):489-501.
[14] 王一宾,程玉胜,何月,等.回归核极限学习机的多标记学习算法[J].模式识别与人工智能,2018,31(5):419-430.(WANG Y B,CHENG Y S,HE Y,et al.Multi-label learning algorithm of regression kernel extreme learning machine[J].Pattern Recognition and Artificial Intelligence,2018,31(5):419-430.)
[15] 张敏灵.一种新型多标记懒惰学习算法[J].计算机研究与发展,2012,49(11):2271-2282.(ZHANG M L.An improved multi-label lazy learning approach[J].Journal of Computer Research and Development,2012,49(11):2271-2282.)
[16] YANG X,HE X.Firefly algorithm:recent advances and applications[J].International Journal of Swarm Intelligence,2013,1(1):36-50.
[17] HIDALGOPANIAGUA A,MIDUEL A V,JOAQUIN F,et al.Solving the multi-objective path planning problem in mobile robotics with a firefly-based approach[J].Soft Computing,2017,21(4):1-16.
[18] LEI Y,ZHAO D,CAI H B.Prediction of length-of-day using extreme learning machine[J].Geodesy and Geodynamics,2015,6(2):151-159.
[19] WANG Z,XIN J,TIAN S,et al.Distributed and weighted extreme learning machine for imbalanced big data learning[J].Tsinghua Science and Technology,2017,22(2):160-173.
[20] LUO F F,GUO W Z,YU Y L,et al.A multi-label classification algorithm based on kernel extreme learning machine[J].Neurocomputing,2017,260:313-320.
[21] 杨明极,马池,王娅,等.一种改进K-means 聚类的FCMM 算法[J/OL].计算机应用研究,2019,36(7)[2018- 04- 12].http://www.arocmag.com/article/02-2019-07-006.html.(YANG M J,MA C,WANG Y,et al.Algorithm named FCMM to improve K-means clustering algorithm[J/OL].Application Research of Computers,2019,36(7)[2018- 04- 12].http://www.arocmag.com/article/02-2019-07-006.html.)
[22] WANG H,WANG W,ZHOU X,et al.Firefly algorithm with neighborhood attraction[J].Information Sciences,2017,382/383:374-387.
[23] 程美英,倪志伟,朱旭辉.萤火虫优化算法理论研究综述[J].计算机科学,2015,42(4):19-24.(CHENG M Y,NI Z W,ZHU X H.Overview on glowworm swarm optimization or firefly algorithm[J].Computer Science,2015,42(4):19-24.)
[24] ZHANG M L,ZHOU Z H.A Review on multi-label learning algorithms[J].IEEE Transactions on Knowledge & Data Engineering,2014,26(8):1819-1837.
[25] DEMSAR J.Statistical comparisons of classifiers over multiple data sets[J].Journal of Machine Learning Research,2006,7(1):1-30.
[26] ZHANG M,WU L.Lift:Multi-label learning with label-specific features[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(1):107-120.
[27] LIN Y,LI Y,WANG C,et al.Attribute reduction for multi-label learning with fuzzy rough set[J].Knowledge-Based Systems,2018,152:51-56.
[2] 何志芬,杨明,刘会东.多标记分类和标记相关性的联合学习[J].软件学报,2014,25(9):1967-1981.(HE Z F,YANG M,LIU H D.Joint learning of multi-label classification and label correlations[J].Journal of Software,2014,25(9):1967-1981.)
[3] LIU J,CHANG W,WU Y,et al.Deep learning for extreme multi-label text classification[C]// Proceedings of the 40th International ACM SIGIR Conference on Research & Development in Information Retrieval.New York:ACM,2017:115-124.
[4] KORDMAHALLEH M M,HOMAIFAR A,DUKKA B K C.Hierarchical multi-label gene function prediction using adaptive mutation in crowding niching[C]// Proceedings of the 13th IEEE International Conference on BioInformatics and BioEngineering.Piscataway,NJ:IEEE,2013:1-6.
[5] ZHU X,LI X,ZHANG S.Block-row sparse multiview multilabel learning for image classification[J].IEEE Transactions on Cybernetics,2016,46(2):450.
[6] WANG Z,CHEN T,LI G,et al.Multi-label image recognition by recurrently discovering attentional regions[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision.Washington,DC:IEEE Computer Society,2017:464-472.
[7] OZONAT K M,YOUNG D E.Towards a universal marketplace over the Web:statistical multi-label classification of service provider forms with simulated annealing[C]// Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York:ACM,2009:1295-1304.
[8] HOU S,ZHOU S,CHEN L,et al.Multi-label learning with label relevance in advertising video[J].Neurocomputing,2016,171(C):932-948.
[9] BOUTELL M R,LUO J,SHEN X,et al.Learning multi-label scene classification [J].Pattern Recognition,2004,37(9):1757-1771.
[10] ZHANG M,ZHOU Z.ML-KNN:a lazy learning approach to multi-label learning[J].Pattern Recognition,2007,40(7):2038-2048.
[11] LEE J,KIM H,KIM N R,et al.An approach for multi-label classification by directed acyclic graph with label correlation maximization[J].Information Sciences,2016,351(C):101-114.
[12] ELISSEEFF A E,WESTON J.A kernel method for multi-labelled classification[C]// Proceedings of the 14th International Conference on Neural Information Processing Systems:Natural and Synthetic.Cambridge,MA:MIT Press,2002:681-687.
[13] HUANG G,ZHU Q,SIEW C K.Extreme learning machine:theory and applications[J].Neurocomputing,2006,70(1/2/3):489-501.
[14] 王一宾,程玉胜,何月,等.回归核极限学习机的多标记学习算法[J].模式识别与人工智能,2018,31(5):419-430.(WANG Y B,CHENG Y S,HE Y,et al.Multi-label learning algorithm of regression kernel extreme learning machine[J].Pattern Recognition and Artificial Intelligence,2018,31(5):419-430.)
[15] 张敏灵.一种新型多标记懒惰学习算法[J].计算机研究与发展,2012,49(11):2271-2282.(ZHANG M L.An improved multi-label lazy learning approach[J].Journal of Computer Research and Development,2012,49(11):2271-2282.)
[16] YANG X,HE X.Firefly algorithm:recent advances and applications[J].International Journal of Swarm Intelligence,2013,1(1):36-50.
[17] HIDALGOPANIAGUA A,MIDUEL A V,JOAQUIN F,et al.Solving the multi-objective path planning problem in mobile robotics with a firefly-based approach[J].Soft Computing,2017,21(4):1-16.
[18] LEI Y,ZHAO D,CAI H B.Prediction of length-of-day using extreme learning machine[J].Geodesy and Geodynamics,2015,6(2):151-159.
[19] WANG Z,XIN J,TIAN S,et al.Distributed and weighted extreme learning machine for imbalanced big data learning[J].Tsinghua Science and Technology,2017,22(2):160-173.
[20] LUO F F,GUO W Z,YU Y L,et al.A multi-label classification algorithm based on kernel extreme learning machine[J].Neurocomputing,2017,260:313-320.
[21] 杨明极,马池,王娅,等.一种改进K-means 聚类的FCMM 算法[J/OL].计算机应用研究,2019,36(7)[2018- 04- 12].http://www.arocmag.com/article/02-2019-07-006.html.(YANG M J,MA C,WANG Y,et al.Algorithm named FCMM to improve K-means clustering algorithm[J/OL].Application Research of Computers,2019,36(7)[2018- 04- 12].http://www.arocmag.com/article/02-2019-07-006.html.)
[22] WANG H,WANG W,ZHOU X,et al.Firefly algorithm with neighborhood attraction[J].Information Sciences,2017,382/383:374-387.
[23] 程美英,倪志伟,朱旭辉.萤火虫优化算法理论研究综述[J].计算机科学,2015,42(4):19-24.(CHENG M Y,NI Z W,ZHU X H.Overview on glowworm swarm optimization or firefly algorithm[J].Computer Science,2015,42(4):19-24.)
[24] ZHANG M L,ZHOU Z H.A Review on multi-label learning algorithms[J].IEEE Transactions on Knowledge & Data Engineering,2014,26(8):1819-1837.
[25] DEMSAR J.Statistical comparisons of classifiers over multiple data sets[J].Journal of Machine Learning Research,2006,7(1):1-30.
[26] ZHANG M,WU L.Lift:Multi-label learning with label-specific features[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(1):107-120.
[27] LIN Y,LI Y,WANG C,et al.Attribute reduction for multi-label learning with fuzzy rough set[J].Knowledge-Based Systems,2018,152:51-56.