基于多目标决策的数据挖掘模型选择研究
|
摘要
数据挖掘是从海量数据中提取潜在有用模式的过程,近年来在银行和通信等多个应用领域取得了显著成果。聚类和分类是数据挖掘中的两个核心技术。针对特定的实际数据挖掘任务,可供选择的常用算法有数十种,进而导致了以下两个问题：缺乏确定的规则帮助人们从众多算法中选择适用于其问题的算法；模型的可靠性、运行效率等很难得到检验。
针对上述问题,本文对数据挖掘中聚类和分类的模型选择问题进行研究：
第一,建立数据挖掘模型选择的理论框架,以实际应用问题为出发点,将业务问题转化为对项目目标及相关数据特征的描述。以数据挖掘应用的问题空间、选定的算法空间和算法性能评价指标空间为基础,通过多目标决策的方法进行模型选择。
第二,从基于划分、层次、密度和模型的聚类算法中选取了六种代表性的算法,根据它们对数据集的划分结果,综合考虑聚类评价的外部准则、内部准则和相对准则中的12个具体评价指标,采用多目标决策方法进行综合比较分析,根据模型选择的结果产生经验规则,形成针对特定业务问题的自动模型选择机制。
第三,从基于决策树、函数方法、贝叶斯理论、懒惰学习和关联规则的分类算法中选取12种代表性算法,以软件缺陷预测为背景,从二元分类和集成学习两个层次,使用多目标决策方法进行模型选择,根据选择的结果一方面形成经验规则；另一方面指导特定问题中数据特征变换和算法的设计。
第四,以分类模型选择的结果为指导,针对不平衡数据学习这一热点问题,从训练集中各类样本分布角度出发,提出了基于密度的过采样算法；从数据特征角度出发,提出了基于重采样和集成技术的特征选择方法,并通过在不平衡数据上与基础分类算法及其它同类算法进行对比,证实了两个方法的有效性。
最后,独立开发了DSOLVER多目标决策工具箱,集成了14种常用的多目标决策方法和技术,通过数据展示、标准化方法、决策分析、敏感性分析、算法对比和群体决策等模块,为实现本文所提出的基于多目标决策的数据挖掘模型选择方法及其它相关决策问题提供了有效的工具。
Data mining is the computational process of discovering patterns in large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics, and database systems. It has achieved very significant results in multiple applications such as banking and communications in recent year. Both clustering and classification are key technologies of data mining. There are dozens of algorithms available for a specific data mining task. However, in practice, data mining techniques are severely limited by two main challenges. One is lack of rules or knowledge to help people to choose proper algorithms for their ongoing data mining project, followed by difficult to test the reliability and the operating efficiency of the selected model.
For the above mentioned problems, this research focus on the model selection of data mining, specifically, on the clustering and classification:
First, the theoretical framework of model selection for data mining is established. The framework disintegrates the model selection problem into tasks space, algorithm space, evaluating criteria space and evaluating strategy. The tasks space contains the description of the ongoing data mining project. The algorithm space defines the available algorithms for the task. Evaluating criteria space contains the metrics used to evaluate the performance of the models. Evaluating strategy describes which and how multiple criteria decision making (MCDM) methods are used in the model ranking and final selection.
Second, five representative clustering algorithms are selected from the division based algorithm, hierarchical algorithm, density based algorithm and model based clustering algorithm. And then constructes the evaluating space with11performance metrics which are from external, internal and relative clustering evaluation criteria. After comprehensive comparative analysis by using the MCDM methods, a mechanism to automatic algorithm selection for a specific data mining task are built.
Third,12typical algorithms are selected as the algorithm space from the decision tree, functions based methods, Bayesian theory, lazy learning and association rule-based classification algorithms. This study examines the model selection problem at the direction of binary classification and ensemble learning for the software defect prediction. This study acquired two main achievements based on the ranking results. On the one hand, empirical knowledge was acquired for the software defect prediction task. On the other hand, the ranking results are used to guide the design of feature transform, new algorithms for specific tasks.
Fourth, guided by the classification models selection results, a density based over sampling algorithm and an ensemble feature selection method for imbalanced learning are proposed. The experimental results indicated that the proposed algorithms are effective for the imbalanced learning problems.
Finally, an open-source MCDM software named DSOLVER for data mining model selection and general decision making problems is developed. The DSOLVER consisted of14MCDM algorithms including the data visualization, standardize processing, decision analysis, sensitivity analysis, results comparison and group decision-making modules.
针对上述问题,本文对数据挖掘中聚类和分类的模型选择问题进行研究：
第一,建立数据挖掘模型选择的理论框架,以实际应用问题为出发点,将业务问题转化为对项目目标及相关数据特征的描述。以数据挖掘应用的问题空间、选定的算法空间和算法性能评价指标空间为基础,通过多目标决策的方法进行模型选择。
第二,从基于划分、层次、密度和模型的聚类算法中选取了六种代表性的算法,根据它们对数据集的划分结果,综合考虑聚类评价的外部准则、内部准则和相对准则中的12个具体评价指标,采用多目标决策方法进行综合比较分析,根据模型选择的结果产生经验规则,形成针对特定业务问题的自动模型选择机制。
第三,从基于决策树、函数方法、贝叶斯理论、懒惰学习和关联规则的分类算法中选取12种代表性算法,以软件缺陷预测为背景,从二元分类和集成学习两个层次,使用多目标决策方法进行模型选择,根据选择的结果一方面形成经验规则；另一方面指导特定问题中数据特征变换和算法的设计。
第四,以分类模型选择的结果为指导,针对不平衡数据学习这一热点问题,从训练集中各类样本分布角度出发,提出了基于密度的过采样算法；从数据特征角度出发,提出了基于重采样和集成技术的特征选择方法,并通过在不平衡数据上与基础分类算法及其它同类算法进行对比,证实了两个方法的有效性。
最后,独立开发了DSOLVER多目标决策工具箱,集成了14种常用的多目标决策方法和技术,通过数据展示、标准化方法、决策分析、敏感性分析、算法对比和群体决策等模块,为实现本文所提出的基于多目标决策的数据挖掘模型选择方法及其它相关决策问题提供了有效的工具。
Data mining is the computational process of discovering patterns in large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics, and database systems. It has achieved very significant results in multiple applications such as banking and communications in recent year. Both clustering and classification are key technologies of data mining. There are dozens of algorithms available for a specific data mining task. However, in practice, data mining techniques are severely limited by two main challenges. One is lack of rules or knowledge to help people to choose proper algorithms for their ongoing data mining project, followed by difficult to test the reliability and the operating efficiency of the selected model.
For the above mentioned problems, this research focus on the model selection of data mining, specifically, on the clustering and classification:
First, the theoretical framework of model selection for data mining is established. The framework disintegrates the model selection problem into tasks space, algorithm space, evaluating criteria space and evaluating strategy. The tasks space contains the description of the ongoing data mining project. The algorithm space defines the available algorithms for the task. Evaluating criteria space contains the metrics used to evaluate the performance of the models. Evaluating strategy describes which and how multiple criteria decision making (MCDM) methods are used in the model ranking and final selection.
Second, five representative clustering algorithms are selected from the division based algorithm, hierarchical algorithm, density based algorithm and model based clustering algorithm. And then constructes the evaluating space with11performance metrics which are from external, internal and relative clustering evaluation criteria. After comprehensive comparative analysis by using the MCDM methods, a mechanism to automatic algorithm selection for a specific data mining task are built.
Third,12typical algorithms are selected as the algorithm space from the decision tree, functions based methods, Bayesian theory, lazy learning and association rule-based classification algorithms. This study examines the model selection problem at the direction of binary classification and ensemble learning for the software defect prediction. This study acquired two main achievements based on the ranking results. On the one hand, empirical knowledge was acquired for the software defect prediction task. On the other hand, the ranking results are used to guide the design of feature transform, new algorithms for specific tasks.
Fourth, guided by the classification models selection results, a density based over sampling algorithm and an ensemble feature selection method for imbalanced learning are proposed. The experimental results indicated that the proposed algorithms are effective for the imbalanced learning problems.
Finally, an open-source MCDM software named DSOLVER for data mining model selection and general decision making problems is developed. The DSOLVER consisted of14MCDM algorithms including the data visualization, standardize processing, decision analysis, sensitivity analysis, results comparison and group decision-making modules.
引文
[1]Han, J., Kamber, M.,& Pei, J. Data mining:concepts and techniques[M]. Morgan Kaufmann, 2006.
[2]Domingos P. Toward knowledge-rich data mining[J]. Data Mining and Knowledge Discovery. 2007,15(1):21-28.
[3]Caruana R, Niculescu-Mizil A. Data mining in metric space:an empirical analysis of supervised learning performance criteria[C]. Proceedings of the tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,2004,69-78.
[4]Nickerson A, Japkowicz N, Milios E. Using unsupervised learning to guide re-sampling in imbalanced data sets[C]. Proceedings of the Eighth International Workshop on AI and Statitsics,2001,261-265.
[5]Fisher DH. Knowledge acquisition via incremental conceptual clustering[J]. Machine Learning.1987,2(2):139-172.
[6]Xu R, Wunsch D. Survey of clustering algorithms[J]. IEEE Transactions on Neural Networks, 2005,16(3):645-678.
[7]Domingos P. The role of Occam's razor in knowledge discovery[J]. Data Mining and Knowledge Discovery,1999,3(4):409-425.
[8]Blumer A, Ehrenfeucht A, Haussler D, et al. Occam's razor[J]. Information Processing Letters, 1987,24(6):377-380.
[9]Macready WG, Wolpert DH. No free lunch theorems for search [R]. Technical Report SFI-TR-95-02-010, Santa Fe Institute,1995.
[10]Wolpert DH, Macready WG. No free lunch theorems for optimization[J]. IEEE Transactions on Evolutionary Computation,1997,1(1):67-82.
[11]Rice JR. The algorithm selection problem[J]. Advances in Computers,1976,15:65-118.
[12]King RD, Feng C, Sutherland A. Statlog:comparison of classification algorithms on large real-world problems[J]. Applied Artificial Intelligence an International Journal,1995,9(3): 289-333.
[13]Rand WM. Objective criteria for the evaluation of clustering methods[J]. Journal of the American Statistical Association,1971,66(336):846-850.
[14]Xie XL, Beni G. A validity measure for fuzzy clustering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1991,13(8):841-847.
[15]Peng Y, Gang K, Wang G, et al. Empirical evaluation of classifiers for software risk management[J]. International Journal of Information Technology & Decision Making,2009, 8(04):749-767.
[16]Peng Y, Kou G, Shi Y. Knowledge-rich data mining in financial risk detection[J]. Computational Science-ICCS,2009,534-542.
[17]Peng Y, Kou G, Wang G, et al. FAMCDM:A fusion approach of MCDM methods to rank multiclass classification algorithms[J]. Omega,2011,39(6):677-689.
[18]Rigopoulos G, Anagnostopoulos K. Fuzzy multicriteria assignment for nominal classification: methodology and application in evaluation of Greek Bank's electronic payment retailers[J]. International Journal of Information Technology & Decision Making,2010,9(03):437-454.
[19]Nie G, Wang G, Zhang P, et al. Finding the hidden pattern of credit card holder's churn:a case of China[J]. Nie G, Wang G, Zhang P, et al. Finding the Hidden Pattern of Credit Card Holder's Churn:A Case of China[C]//Proceedings of the 9th International Conference on Computational Science, Springer-Verlag,2009:561-569.
[20]Fayyad U, Piatetsky-Shapiro G, Smyth P. The KDD process for extracting useful knowledge from volumes of data[J]. Communications of the ACM,1996,39(11):27-34.
[21]Fayyad U, Piatetsky-Shapiro G, Smyth P. From data mining to knowledge discovery in databases[J]. Al Magazine,1996,17(3):37.
[22]Bhattacharyya S, Jha S, Tharakunnel K, et al. Data mining for credit card fraud:a comparative study[J]. Decision Support Systems,2011,50(3):602-613.
[23]Ling CX, Li C. Data mining for direct marketing:problems and solutions[C]. Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining,1998:73-79.
[24]Witten IH, Frank E. Data mining:practical machine learning tools and techniques[M]. Morgan Kaufmann,2005.
[25]Chapman P, Clinton J, Kerber R, et al. CRISP-DM 1.0 Step-by-step data mining guide[OL]. http://www.crisp-dm.org.
[26]范明,柴玉梅,咎红英.统计学习基础-数据挖掘,推理与预测[M].北京：电子工业出版社,2007.
[27]Yong S, Peng Y, GANG K, et al. Classifying credit card accounts for business intelligence and decision making:A multiple-criteria quadratic programming approach[J]. International Journal of Information Technology & Decision Making,2005,4(04):581-599.
[28]Kou C,Peng Y, Chen Z, et al. Multiple criteria mathematical programming for multi-class classification and application in network intrusion detection[J]. Information Sciences,2009, 179(4):371-381.
[29]Peng Y, Kou G, Sabatka A, et al. Application of classification methods to individual disability income insurance fraud detection[J]. Computational Science-ICCS 2007,852-858.
[30]Peng Y, Wang G, Kou G, Shi Y. An empirical study of classification algorithm evaluation for financial risk prediction[J]. Applied Soft Computing,2011,11(2):2906-2915.
[31]Peng Y, Wang G, Wang H. User preferences based software defect detection algorithms selection using MCDM[J]. Information Sciences,2012,191:3-13.
[32]Provost F, Domingos P. Tree induction for probability-based ranking[J]. Machine Learning, 2003,52(3):199-215.
[33]Ren N, Zargham M, Rahimi S. A decision tree-based classification approach to rule extraction for security analysis[J]. International Journal of Information Technology & Decision Making, 2006,5(01):227-240.
[34]Cieslak DA, Hoens TR, Chawla NV, et al. Hellinger distance decision trees are robust and skew-insensitive[J]. Data Mining and Knowledge Discovery,2011,24(1):136-158.
[35]Porter AA, Selby RW. Evaluating techniques for generating metric-based classification trees[J]. Journal of Systems and Software,1990,12(3):209-218.
[36]Todorovski L, Dzeroski S. Combining classifiers with meta decision trees[J]. Machine Learning,2003,50(3):223-249.
[37]Quinlan JR. C4.5:programs for machine learning[M]. Morgan kaufmann,1993.
[38]Breiman L, Friedman J, Stone CJ, Olshen RA. Classification and regression trees[M]. Chapman & Hall/CRC,1984.
[39]Adam A, Shapiai MI, Ibrahim Z, et al. Artificial neural network-Naive bayes fusion for solving classification problem of imbalanced dataset[C]. IEEE 4th International Conference on Modeling, Simulation and Applied Optimization (ICMSAO),2011,1-5.
[40]Domingos P, Pazzani M. On the optimality of the simple Bayesian classifier under zero-one loss[J]. Machine Learning,1997,29(2):103-130.
[4]] Kohavi R. Scaling up the accuracy of naive-Bayes classifiers:A decision-tree hybrid[C]. Proceedings of the second International Conference on Knowledge Discovery and Data Mining,1996,7:1-6.
[42]Ting KM, Zheng Z. A study of ADABOOST with naive Bayesian classifiers:weakness and improvement[J]. Computational Intelligence,2003,19(2):186-200.
[43]Zadrozny B, Elkan C. Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers[C]. In Proceedings of the Eighteenth International Conference on Machine Learning.2001,609-616..
[44]Gao Y, Wang J. Active learning method of Bayesian networks classifier based on cost-sensitive sampling[C].2011 IEEE International Conference on Computer Science and Automation Engineering (CSAE),2011,233-236.
[45]Hughes WR. A statistical framework for strategic decision making with AHP:Probability assessment and Bayesian revision[J]. Omega,2009,37(2):463-470.
[46]Vapnik V. The nature of statistical learning theory[M]. Springer,1999.
[47]Wan H L, Chowdhury M. Image semantic classification by using SVM[J]. Journal of software, 2012,14(11):1891-1899.
[48]Wang Z, Crammer K, Vucetic S. Breaking the curse of kernelization:Budgeted stochastic gradient descent for large-scale svm training [J]. Journal of Machine Learning Research,2012, 13:3103-3131.
[49]Geebelen D, Suykens J A K, Vandewalle J. Reducing the number of support vectors of SVM classifiers using the smoothed separable case approximation [J]. IEEE Transactions on Neural Networks and Learning Systems,2012,23(4):682-688..
[50]Ren J. ANN vs. SVM:Which one performs better in classification of MCCs in mammogram imaging [J]. Knowledge-Based Systems,2012,26:144-153.
[51]Niu X X, Suen C Y. A novel hybrid CNN-SVM classifier for recognizing handwritten digits[J]. Pattern Recognition,2012,45(4):1318-1325.
[52]Alejo R, Valdovinos RM, Garcia V, et al. A hybrid method to face class overlap and class imbalance on neural networks and multi-class scenarios[J]. Pattern Recognition Letters,2013, 34(4):380-388.
[53]Batuwita R, Palade V. Efficient resampling methods for training support vector machines with imbalanced datasets[C]. The 2010 International Joint Conference on Neural Networks (IJCNN),2010,1-8.
[54]Chen K, Xu L, Chi H. Improved learning algorithms for mixture of experts in multiclass classification[J]. Neural Networks,1999,12(9):1229-1252.
[55]Lu X, Tang K, Yao X. Evolving neural networks with maximum AUC for imbalanced data classification[J]. Hybrid Artificial Intelligence Systems,2010:335-342.
[56]Barker TJ, Zabinsky ZB. A multicriteria decision making model for reverse logistics using analytical hierarchy process[J]. Omega,2011,39(5):558-573.
[57]Le Cessie S, Van Houwelingen J. Ridge estimators in logistic regression[J]. Applied Statistics, 1992:191-201.
[58]Liu W, Chawla S. Class confidence weighted kNN algorithms for imbalanced data sets[J]. Advances in Knowledge Discovery and Data Mining,2011:345-356.
[59]Dasarathy BV. Nearest neighbor (NN) norms:NN pattern classification techniques[M]. IEEE Computer Society,1991.
[60]Kriminger E, Principe J, Lakshminarayan C. Nearest neighbor distributions for imbalanced classification[C]. The 2012 International Joint Conference on Neural Networks, IEEE,2012, 1-5.
[61]Yuxuan L, Zhang X. Improving k nearest neighbor with exemplar generalization for imbalanced classification[C].15th Pacific-Asia Conference, PAKDD 2011, Springer,2011, 1-12.
[62]Breiman L. Bagging predictors[J]. Machine Learning,1996,24(2):123-140.
[63]Buhlmann P. Bagging, boosting and ensemble methods[M]. Handbook of Computational Statistics, Springer,2012,985-1022.
[64]Galar M, Fernandez A, Barrenechea E, et al. A review on ensembles for the class imbalance problem:bagging-, boosting-, and hybrid-based approaches[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C:Applications and Reviews,2012,42(4):463-484.
[65]Dietterich TG. An experimental comparison of three methods for constructing ensembles of decision trees:Bagging, boosting, and randomization[J]. Machine Learning,2000,40(2): 139-157.
[66]Xie Y, Li X. Churn prediction with Linear Discriminant Boosting algorithm[C]. International* Conference on Machine Learning and Cybernetics, IEEE,2008,228-233.
[67]Sheng C, Haibo H, Garcia EA. RAMOBoost:ranked minority oversampling in boosting[J]. IEEE Transactions on Neural Networks,2010,21(10):1624-1642.
[68]Nakhaeizadeh G, Schnabl A. Development of multi-criteria metrics for evaluation of data mining algorithms[C]. Third International Conference on Knowledge Discovery and Data Mining,1997,37-42.
[69]Challagulla Vub, Farokh Bb, Yen I-L, et al. Empirical assessment of machine learning based software defect prediction techniques[J]. International Journal on Artificial Intelligence Tools, 2008,17(2):389-400.
[70]Hartigan JA. Clustering algorithms[M]. John Wiley & Sons,1975.
[71]Jain AK, Dubes RC. Algorithms for clustering data[M]. Prentice-Hall,1988.
[72]Jain AK, Murty MN, Flynn PJ. Data clustering:a review[J]. ACM Computing Surveys,1999, 31(3):264-323.
[73]Hartigan JA, Wong MA. Algorithm AS 136:A k-means clustering algorithm[J]. Applied Statistics,1979:100-108.
[74]Birant D, Kut A. ST-DBSCAN:an algorithm for clustering spatial-temporal data[J]. Data & Knowledge Engineering,2007,60(1):208-221.
[75]Guha S, Rastogi R, Shim K. CURE:an efficient clustering algorithm for large databases[C]. ACM SIGMOD Record, ACM,1998,73-84.
[76]Livny TZRRM. BIRCH:an efficient data clustering method for very large databases[C]. ACM SIGMOD International Conference on Management of Data,1996,103-114.
[77]Liu M, Li F, Wu C. The order structure of fuzzy numbers based on the level characteristics and its application to optimization problems[J]. Science in China Series F:Information Sciences, 2002,45(6):433-441.
[78]Banfield JD, Raftery AE. Model-based Gaussian and non-Gaussian clustering[J]. Biometrics. 1993:803-821.
[79]McLachlan GJ, Basford KE. Mixture models:inference and applications to clustering[J]. Statistics:Textbooks and Monographs, New York:Dekker,1988,1.
[80]Steinbach M, Karypis G, Kumar V. A comparison of document clustering techniques[C]. KDD Workshop on Text Mining, Boston,2000,525-526.
[81]Fecker D, Margner V, Fingscheidt T. Training of classifiers for quality control of on-line laser brazing processes with highly imbalanced datasets[J]. Pattern Recognition,2012:367-376.
[82]Airman El, Avery RB, Eisenbeis RA, et al. Application of classification techniques in business, banking and finance[M]. JAI press,1981.
[83]Japkowicz N. The class imbalance problem:significance and strategies[C]. Proceedings of the 2000 International Conference on Artificial Intelligence,2000,1:111-117.
[84]Chawla NV, Japkowicz N, Kotcz A. Editorial:special issue on learning from imbalanced data sets[J]. ACM SIGKDD Explorations Newsletter,2004,6(1):1-6.
[85]Chawla NV, Bowyer KW, Hall LO, et al. SMOTE:synthetic minority over-sampling technique[J]. Journal of Artificial Intelligence Research,2002,16:321-357.
[86]Chawla NV. Data mining for imbalanced datasets:an overview[J]. Data Mining and Knowledge Discovery Handbook,2010:875-886.
[87]He H, Garcia EA. Learning from imbalanced data[J]. IEEE Transactions on Knowledge and Data Engineering,2009,21(9):1263-1284.
[88]Abbasnejad ME, Ramachandram D, Mandava R. A survey of the state of the art in learning the kernels[J]. Knowledge and Information Systems,2011,31 (2):193-221.
[89]吴克寿,曾志强.非平衡数据集分类研究[J].计算机技术与发展.2011,21(9)：39-42.
[90]马胜祥,马建庆,杨明.基于核函数拟合的非平衡数据分类方法[J].计算机应用与软件,2010,27(004)：177-179.
[91]李正欣,赵林度.基于SMOTEBoost的非均衡数据集SVM分类器[J].系统工程,2008,26(5)：116-119.
[92]岳超源,系统工程.决策理论与方法[M].北京：科学出版社,2003.
[93]王国华,梁樑.决策理论与方法[M].合肥：中国科学技术大学出版社,2006.
[94]徐玖平,李军.多目标决策的理论与方法[M].北京：清华大学出版社有限公司,2005.
[95]陈埏.决策分析[M].北京：科学出版杜.1987.
[96]高洪深.决策支持系统(DSS)理论·方法·案例[M].北京：清华大学出版社,2005.
[97]杨雷.群体决策理论与应用：群体决策中的个体偏好集结方法研究[M].北京：经济科学 出版社,2004.
[98]杨善林.智能决策方法与智能决策支持系统[M].北京：科学出版社,2005.
[99]吴祈宗,朱心想.几种区间数判断矩阵排序权向量计算方法的比较研究[J].北京工商大学学报：自然科学版,2002,20(4)：53-57.
[100]郭显光.多指标综合评价中权数的确定[J].数量经济技术经济研究,1989,11(6)：49-52.
[101]陆明生.多目标决策中的权系数[J].系统工程理论与实践,1986,6(4)：77-78.
[102]王宗军.多目标权系数赋值方法及其选择策略[J].系统工程与电子技术,1993,15(6)：35-41.
[103]吴云燕,华中生,查勇.AHP中群决策权重的确定与判断矩阵的合并[J].运筹与管理,2003,12(4)：16-21.
[104]梁樑,熊立,王国华.一种群决策中专家客观权重的确定方法[J].系统工程与电子技术,2005,27(4)：652-655.
[105]夏洪胜,盛昭瀚,徐南荣.多层多目标决策方法的综述[J].系统工程与电子技术,1992,7：27-32.
[106]顾基发.多目标决策问题[J],自然杂志,1980,2：14.
[107]Hwang CL, Yoon K. Multiple attribute decision making:methods and applications:a state-of-the-art survey[M]. Springer-Verlag New York,1981.
[108]Abo-Sinna MA, Amer AH. Extensions of TOPSIS for multi-objective large-scale nonlinear programming problems[J]. Applied Mathematics and Computation,2005,162(1):243-256.
[109]Opricovic S, Tzeng GH. Compromise solution by MCDM methods:a comparative analysis of VIKOR and TOPSIS[J]. European Journal of Operational Research,2004,156(2):445-455.
[110]张建胜,吴祈宗TOPSIS方法在Vague集中的应用[J].北京理工大学学报,2006,26(10)：937-940.
[111]周晓光,张强,胡望斌.基于Vague集的TOPSIS方法及其应用[J].系统工程理论方法应用,2005,14(6)：537-541.
[112]Brans JP. L'ingenierie de la decision:I'elaboration d'instruments d'aide a la decision[M]. Universite Laval, Faculte des sciences de I'administration,1982.
[113]Brans JP, Mareschal B. How to Decide with PROMETHEE[J]. ULB and VUB Brussels Universities Vrije Universiteit Brussel,1986,254-258.
[114]Brans JP, Mareschal B. PROMETHEE methods[J]. Multiple Criteria Decision Analysis:State of the Art Surveys,2005:163-186.
[115]邓乐斌,梅晓玲.信息不完全确定的PROMETHEE/TOPSIS方法及应用[J].长江大学学报(自科版),2007,4(4)：36-37.
[116]王建军,杨德礼.信息系统外包决策的AHP与PROMETHEE方法[J].管理学报,2006,3(3)：287-291.
[117]Chu MT, Shyu J, Tzeng GH, et al. Comparison among three analytical methods for knowledge communities group-decision analysis[J]. Expert Systems with Applications,2007,33(4): 1011-1024.
[118]Opricovic S, Tzeng GH. Multicriteria planning of post-earthquake sustainable seconstruction[J]. Computer-Aided Civil and Infrastructure Engineering,2002,17(3): 211-220.
[119]Opricovic S. Multicriteria optimization of civil engineering systems[J]. Faculty of Civil Engineering, Belgrade,1998,2(1):5-21.
[120]Saaty TL. What is the analytic hierarchy process?[M]. Springer,1988.
[121]华中生,吴云燕,徐晓燕.一种AHP判断矩阵一致性调整的新方法[J].系统工程与电子技术,2003,25(1)：38-40.
[122]黄德才,郑河荣.AHP方法中判断矩阵的标度扩展构造法[J].系统工程,2003,21(1)：105-109.
[123]王建,黄凤岗,景韶光.AHP中判断矩阵一致性调整方法研究[J].系统工程理论与实践,2005,25(8)：85-91.
[124]肖四汉,樊治平,王梦光.群决策中两类偏好信息-AHP判断矩阵和模糊偏好关系矩阵的一致化方法[J].系统工程学报,2002,17(1)：82-86.
[125]Cooper WW, Seiford LM, Zhu J. Data envelopment analysis:History, models, and interpretations[J]. Handbook on data envelopment analysis,2011:1-39.
[126]马占新.数据包络分析方法的研究进展[J].系统工程与电子技术,2002,24(3)：42-46.
[127]魏权龄.数据包络分析(DEA)[J].科学通报,2000,45(17)：1793-1808.
[128]魏权龄.数据包络分析[M].北京：科学出版社,2004.
[129]Roy B. Classement et choix en presence de points de vue multiples[J]. RAIRO-Operations Research-Recherche Operationnelle,1968,2(V1):57-75.
[130]Figueira J, Mousseau V, Roy B. ELECTRE methods[J]. Multiple Criteria Decision Analysis: State of the Art Surveys,2005:133-153.
[131]Milani A, Shanian A, El-Lahham C. Using different ELECTRE methods in strategic planning in the presence of human behavioral resistance[J]. Advances in Decision Sciences,2006,1-19.
[132]龚本刚,华中生,檀大水.一种语言评价信息不完全的多属性群决策方法[J].中国管理科学,2007,15(1)：88-93.
[133]王坚强.模糊多准则决策方法研究综述[J].控制与决策,2008,23(6)：601-607.
[134]徐泽水.基于语言信息的决策理论与方法[M].北京：科学出版社,2008.
[135]徐玖平,陈建中.群决策理论与方法及实现[M].北京：清华大学出版社,2009.
[136]邓聚龙.灰色系统基本方法(第一版)[M].湖北：华中工学院山版社,1987.
[137]吕锋,崔晓辉.多目标决策灰色关联投影法及其应用[J].系统工程理论与实践,2002, 22(1)：103-107.
[138]赵艳林,杨绿峰,吕海波,等.不确定信息条件下的灰色模式识别[J].控制与决策,2003,18(5)：593-596.
[139]Doyle JR, Green RH. Cross-evaluation in DEA:improving discrimination among DMUs[J]. Infor-Information Systems and Operational Research,1995,33(3):205.
[140]Green RH, Doyle JR, Cook WD. Preference voting and project ranking using DEA and cross-evaluation[J]. European Journal of Operational Research,1996,90(3):461-472.
[141]Liang L, Wu J, Cook WD, Zhu J. Alternative secondary goals in DEA cross-efficiency evaluation[J]. International Journal of Production Economics,2008,113(2):1025-1030.
[142]樊治平,肖四汉.基于OWA算子的不同形式偏好信息的群决策方法[J].控制与决策,2001,16(B11)：749-752.
[143]徐泽水.一种不确定型OWA算子及其在群决策中的应用[J].东南大学学报：自然科学版,2002,32(1)：147-150.
[144]Yager RR. Applications and extensions of OWA aggregations[J]. International Journal of Man-Machine Studies.1992,37(1):103-122.
[145]陈衍泰,陈国宏,李美娟.综合评价方法分类及研究进展[J].管理科学学报,2004,7(2)：69-79.
[146]杜栋,庞庆华.现代综合评价方法与案例精选[M].北京：清华大学出版社有限公司,2005.
[147]Ephraim Y, Malah D. Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator[J]. IEEE Transactions on Acoustics, Speech and Signal Processing,1984,32(6):1109-1121.
[148]Barron A, Rissanen J, Yu B. The minimum description length principle in coding and modeling[J]. IEEE Transactions on Information Theory,1998,44(6):2743-2760.
[149]Grunwald PD. The minimum description length principle[M]. The MIT Press,2007.
[150]李军.不平衡数据学习的研究[D].吉林：吉林大学,2011.
[151]Rendell L, Cho H. Empirical learning as a function of concept character[J]. Machine Learning, 1990,5(3):267-298.
[152]Aha DW. Generalizing from case studies:a case study[C]. Proceedings of the Ninth International Conference on Machine Learning,1992,1-10.
[153]Brodley CE. Addressing the selective superiority problem:automatic algorithm/model class selection[C]. Proceedings of the Tenth International Conference on Machine Learning Citeseer, 1993,17-24.
[154]Lindner G, Studer R. AST:support for algorithm selection with a CBR approach[M]. Principles of Data Mining and Knowledge Discovery, Springer,199,418-423.
[155]Smith-Miles KA. Cross-disciplinary perspectives on meta-learning for algorithm selection[J]. ACM Computing Surveys,2008,41(1):6-15.
[156]UCI Machine Learning Repository[DB]. http://archive.ics.uci.edu/ml/.
[157]Bensusan H, Giraud-Carrier C, Kennedy C. A higher-order approach to meta-learning[C]. Proceedings of the Work-in-Progress Track at the 10th International Conference on Inductive Logic Programming,2000,33-42.
[158]Seewald AK, Petrak J, Widmer G. Hybrid decision tree learners with alternative leaf classifiers: An empirical study[C]. Proceedings of the Fourteenth International Florida Artificial Intelligence Research Society Conference, AAAI Press,2001,407-411.
[159]Todorovski L, Blockeel H, Dzeroski S. Ranking with predictive clustering trees[M]. Springer, 2002.
[160]Ali S, Smith KA. On learning algorithm selection for classification[J]. Applied Soft Computing,2006,6(2):119-138.
[161]Ali S, Smith-Miles KA. A meta-learning approach to automatic kernel selection for support vector machines[J]. Neurocomputing,2006,70(1):173-186.
[162]Castiello C, Castellano G, Fanelli AM. Meta-data:characterization of input features for meta-learning[M]. Modeling Decisions for Artificial Intelligence:Springer,2005,457-468.
[163]Ler D, Koprinska 1, Chawla S. Utilising regression-based landmarkers within a meta-learning framework for algorithm selection[C].22nd Annual International Conference on Machine Learning, Bonn 2005,139-147.
[164]Peterson AH, Martinez T. Estimating the potential for combining learning models[C].22nd Annual International Conference on Machine Learning, Bonn 2005,68-75.
[165]Prodromidis A, Chan P, Stolfo S. Meta-learning in distributed data mining systems:issues and approaches[J]. Advances in Distributed and Parallel Knowledge Discovery,2000,3:81-118.
[166]Kopf C, Taylor C, Keller J. Meta-analysis:from data characterisation for meta-learning to meta-regression[C]. Proceedings of the PKDD-00 Workshop on Data Mining, Decision Support, Meta-Learning and ILP, Citeseer,2000,54-67.
[167]Kuba P, Brazdil P, Soares C, Woznica A. Exploiting sampling and meta-learning for parameter setting support vector machines[C]. Proceedings of the IBERAMIA,2002,217-225.
[168]Soares C, Brazdil PB, Kuba P. A meta-learning method to select the kernel width in support vector regression[J]. Machine Learning,2004,54(3):195-209.
[169]Arinze B, Kim S-L, Anandarajan M. Combining and selecting forecasting models using rule based induction[J]. Computers & Operations Research,1997,24(5):423-433.
[170]Wang X, Smith K, Hyndman R. Characteristic-based clustering for time series data[J]. Data Mining and Knowledge Discovery,2006,13(3):335-364.
[171]Kou G, Peng Y, Shi Y, et al. Discovering credit card holders' behavior by multiple criteria linear programming [J]. Annals of Operations Research,2005,135(1):261-274..
[172]Kwak W, Shi Y, Eldridge S W, et al. Bankruptcy prediction for Japanese firms:using Multiple Criteria Linear Programming data mining approach [J]. International Journal of Business Intelligence and Data Mining,2006,1 (4):401-416.
[173]Kwak W, Shi Y, Kou G. Bankruptcy prediction for Korean firms after the 1997 financial crisis: using a multiple criteria linear programming data mining approach [J]. Review of Quantitative Finance and Accounting,2012,38(4):441-453.
[174]Chapman M, Callis P, Jackson W. Metrics Data Program[OL], NASA IV and V Facility. http://mdp.ivv.nasa.gov/,2004
[175]Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C. DBSMOTE:density based synthetic minority over-sampling technique[J]. Applied Intelligence,2011,36(3):664-684.
[176]Chawla N, Lazarevic A, Hall L, et al. SMOTEBoost:improving prediction of the minority class in boosting[J]. Knowledge Discovery in Databases:PKDD 2003,2003,107-119.
[177]Ramentol E, Caballero Y, Bello R, et al. SMOTE-RSB:a hybrid preprocessing approach based on oversampling and undersampling for high imbalanced data-sets using SMOTE and rough sets theory[J]. Knowledge and Information Systems,2012,33(2):245-265.
[178]Chen X, Wasikowski M. Fast:a roc-based feature selection metric for small samples and imbalanced data classification problems[C]. Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining,2008,124-132.
[179]Kumar PG, Bell JBB. Using continuous feature selection metrics to suppress the class imbalance problem[J]. International Journal of Scientific & Engineering Research,2012, 3(3):1-38.
[180]Zheng Z, Wu X, Srihari R. Feature selection for text categorization on imbalanced data[J]. ACM SIGKDD Explorations Newsletter,2004,6(1):80-89.
[181]Clemencon S, Vayatis N. Nonparametric estimation of the precision-recall curve[C]. Proceedings of the 26th Annual International Conference on Machine Learning, ACM,2009, 185-192.
[2]Domingos P. Toward knowledge-rich data mining[J]. Data Mining and Knowledge Discovery. 2007,15(1):21-28.
[3]Caruana R, Niculescu-Mizil A. Data mining in metric space:an empirical analysis of supervised learning performance criteria[C]. Proceedings of the tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,2004,69-78.
[4]Nickerson A, Japkowicz N, Milios E. Using unsupervised learning to guide re-sampling in imbalanced data sets[C]. Proceedings of the Eighth International Workshop on AI and Statitsics,2001,261-265.
[5]Fisher DH. Knowledge acquisition via incremental conceptual clustering[J]. Machine Learning.1987,2(2):139-172.
[6]Xu R, Wunsch D. Survey of clustering algorithms[J]. IEEE Transactions on Neural Networks, 2005,16(3):645-678.
[7]Domingos P. The role of Occam's razor in knowledge discovery[J]. Data Mining and Knowledge Discovery,1999,3(4):409-425.
[8]Blumer A, Ehrenfeucht A, Haussler D, et al. Occam's razor[J]. Information Processing Letters, 1987,24(6):377-380.
[9]Macready WG, Wolpert DH. No free lunch theorems for search [R]. Technical Report SFI-TR-95-02-010, Santa Fe Institute,1995.
[10]Wolpert DH, Macready WG. No free lunch theorems for optimization[J]. IEEE Transactions on Evolutionary Computation,1997,1(1):67-82.
[11]Rice JR. The algorithm selection problem[J]. Advances in Computers,1976,15:65-118.
[12]King RD, Feng C, Sutherland A. Statlog:comparison of classification algorithms on large real-world problems[J]. Applied Artificial Intelligence an International Journal,1995,9(3): 289-333.
[13]Rand WM. Objective criteria for the evaluation of clustering methods[J]. Journal of the American Statistical Association,1971,66(336):846-850.
[14]Xie XL, Beni G. A validity measure for fuzzy clustering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1991,13(8):841-847.
[15]Peng Y, Gang K, Wang G, et al. Empirical evaluation of classifiers for software risk management[J]. International Journal of Information Technology & Decision Making,2009, 8(04):749-767.
[16]Peng Y, Kou G, Shi Y. Knowledge-rich data mining in financial risk detection[J]. Computational Science-ICCS,2009,534-542.
[17]Peng Y, Kou G, Wang G, et al. FAMCDM:A fusion approach of MCDM methods to rank multiclass classification algorithms[J]. Omega,2011,39(6):677-689.
[18]Rigopoulos G, Anagnostopoulos K. Fuzzy multicriteria assignment for nominal classification: methodology and application in evaluation of Greek Bank's electronic payment retailers[J]. International Journal of Information Technology & Decision Making,2010,9(03):437-454.
[19]Nie G, Wang G, Zhang P, et al. Finding the hidden pattern of credit card holder's churn:a case of China[J]. Nie G, Wang G, Zhang P, et al. Finding the Hidden Pattern of Credit Card Holder's Churn:A Case of China[C]//Proceedings of the 9th International Conference on Computational Science, Springer-Verlag,2009:561-569.
[20]Fayyad U, Piatetsky-Shapiro G, Smyth P. The KDD process for extracting useful knowledge from volumes of data[J]. Communications of the ACM,1996,39(11):27-34.
[21]Fayyad U, Piatetsky-Shapiro G, Smyth P. From data mining to knowledge discovery in databases[J]. Al Magazine,1996,17(3):37.
[22]Bhattacharyya S, Jha S, Tharakunnel K, et al. Data mining for credit card fraud:a comparative study[J]. Decision Support Systems,2011,50(3):602-613.
[23]Ling CX, Li C. Data mining for direct marketing:problems and solutions[C]. Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining,1998:73-79.
[24]Witten IH, Frank E. Data mining:practical machine learning tools and techniques[M]. Morgan Kaufmann,2005.
[25]Chapman P, Clinton J, Kerber R, et al. CRISP-DM 1.0 Step-by-step data mining guide[OL]. http://www.crisp-dm.org.
[26]范明,柴玉梅,咎红英.统计学习基础-数据挖掘,推理与预测[M].北京：电子工业出版社,2007.
[27]Yong S, Peng Y, GANG K, et al. Classifying credit card accounts for business intelligence and decision making:A multiple-criteria quadratic programming approach[J]. International Journal of Information Technology & Decision Making,2005,4(04):581-599.
[28]Kou C,Peng Y, Chen Z, et al. Multiple criteria mathematical programming for multi-class classification and application in network intrusion detection[J]. Information Sciences,2009, 179(4):371-381.
[29]Peng Y, Kou G, Sabatka A, et al. Application of classification methods to individual disability income insurance fraud detection[J]. Computational Science-ICCS 2007,852-858.
[30]Peng Y, Wang G, Kou G, Shi Y. An empirical study of classification algorithm evaluation for financial risk prediction[J]. Applied Soft Computing,2011,11(2):2906-2915.
[31]Peng Y, Wang G, Wang H. User preferences based software defect detection algorithms selection using MCDM[J]. Information Sciences,2012,191:3-13.
[32]Provost F, Domingos P. Tree induction for probability-based ranking[J]. Machine Learning, 2003,52(3):199-215.
[33]Ren N, Zargham M, Rahimi S. A decision tree-based classification approach to rule extraction for security analysis[J]. International Journal of Information Technology & Decision Making, 2006,5(01):227-240.
[34]Cieslak DA, Hoens TR, Chawla NV, et al. Hellinger distance decision trees are robust and skew-insensitive[J]. Data Mining and Knowledge Discovery,2011,24(1):136-158.
[35]Porter AA, Selby RW. Evaluating techniques for generating metric-based classification trees[J]. Journal of Systems and Software,1990,12(3):209-218.
[36]Todorovski L, Dzeroski S. Combining classifiers with meta decision trees[J]. Machine Learning,2003,50(3):223-249.
[37]Quinlan JR. C4.5:programs for machine learning[M]. Morgan kaufmann,1993.
[38]Breiman L, Friedman J, Stone CJ, Olshen RA. Classification and regression trees[M]. Chapman & Hall/CRC,1984.
[39]Adam A, Shapiai MI, Ibrahim Z, et al. Artificial neural network-Naive bayes fusion for solving classification problem of imbalanced dataset[C]. IEEE 4th International Conference on Modeling, Simulation and Applied Optimization (ICMSAO),2011,1-5.
[40]Domingos P, Pazzani M. On the optimality of the simple Bayesian classifier under zero-one loss[J]. Machine Learning,1997,29(2):103-130.
[4]] Kohavi R. Scaling up the accuracy of naive-Bayes classifiers:A decision-tree hybrid[C]. Proceedings of the second International Conference on Knowledge Discovery and Data Mining,1996,7:1-6.
[42]Ting KM, Zheng Z. A study of ADABOOST with naive Bayesian classifiers:weakness and improvement[J]. Computational Intelligence,2003,19(2):186-200.
[43]Zadrozny B, Elkan C. Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers[C]. In Proceedings of the Eighteenth International Conference on Machine Learning.2001,609-616..
[44]Gao Y, Wang J. Active learning method of Bayesian networks classifier based on cost-sensitive sampling[C].2011 IEEE International Conference on Computer Science and Automation Engineering (CSAE),2011,233-236.
[45]Hughes WR. A statistical framework for strategic decision making with AHP:Probability assessment and Bayesian revision[J]. Omega,2009,37(2):463-470.
[46]Vapnik V. The nature of statistical learning theory[M]. Springer,1999.
[47]Wan H L, Chowdhury M. Image semantic classification by using SVM[J]. Journal of software, 2012,14(11):1891-1899.
[48]Wang Z, Crammer K, Vucetic S. Breaking the curse of kernelization:Budgeted stochastic gradient descent for large-scale svm training [J]. Journal of Machine Learning Research,2012, 13:3103-3131.
[49]Geebelen D, Suykens J A K, Vandewalle J. Reducing the number of support vectors of SVM classifiers using the smoothed separable case approximation [J]. IEEE Transactions on Neural Networks and Learning Systems,2012,23(4):682-688..
[50]Ren J. ANN vs. SVM:Which one performs better in classification of MCCs in mammogram imaging [J]. Knowledge-Based Systems,2012,26:144-153.
[51]Niu X X, Suen C Y. A novel hybrid CNN-SVM classifier for recognizing handwritten digits[J]. Pattern Recognition,2012,45(4):1318-1325.
[52]Alejo R, Valdovinos RM, Garcia V, et al. A hybrid method to face class overlap and class imbalance on neural networks and multi-class scenarios[J]. Pattern Recognition Letters,2013, 34(4):380-388.
[53]Batuwita R, Palade V. Efficient resampling methods for training support vector machines with imbalanced datasets[C]. The 2010 International Joint Conference on Neural Networks (IJCNN),2010,1-8.
[54]Chen K, Xu L, Chi H. Improved learning algorithms for mixture of experts in multiclass classification[J]. Neural Networks,1999,12(9):1229-1252.
[55]Lu X, Tang K, Yao X. Evolving neural networks with maximum AUC for imbalanced data classification[J]. Hybrid Artificial Intelligence Systems,2010:335-342.
[56]Barker TJ, Zabinsky ZB. A multicriteria decision making model for reverse logistics using analytical hierarchy process[J]. Omega,2011,39(5):558-573.
[57]Le Cessie S, Van Houwelingen J. Ridge estimators in logistic regression[J]. Applied Statistics, 1992:191-201.
[58]Liu W, Chawla S. Class confidence weighted kNN algorithms for imbalanced data sets[J]. Advances in Knowledge Discovery and Data Mining,2011:345-356.
[59]Dasarathy BV. Nearest neighbor (NN) norms:NN pattern classification techniques[M]. IEEE Computer Society,1991.
[60]Kriminger E, Principe J, Lakshminarayan C. Nearest neighbor distributions for imbalanced classification[C]. The 2012 International Joint Conference on Neural Networks, IEEE,2012, 1-5.
[61]Yuxuan L, Zhang X. Improving k nearest neighbor with exemplar generalization for imbalanced classification[C].15th Pacific-Asia Conference, PAKDD 2011, Springer,2011, 1-12.
[62]Breiman L. Bagging predictors[J]. Machine Learning,1996,24(2):123-140.
[63]Buhlmann P. Bagging, boosting and ensemble methods[M]. Handbook of Computational Statistics, Springer,2012,985-1022.
[64]Galar M, Fernandez A, Barrenechea E, et al. A review on ensembles for the class imbalance problem:bagging-, boosting-, and hybrid-based approaches[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C:Applications and Reviews,2012,42(4):463-484.
[65]Dietterich TG. An experimental comparison of three methods for constructing ensembles of decision trees:Bagging, boosting, and randomization[J]. Machine Learning,2000,40(2): 139-157.
[66]Xie Y, Li X. Churn prediction with Linear Discriminant Boosting algorithm[C]. International* Conference on Machine Learning and Cybernetics, IEEE,2008,228-233.
[67]Sheng C, Haibo H, Garcia EA. RAMOBoost:ranked minority oversampling in boosting[J]. IEEE Transactions on Neural Networks,2010,21(10):1624-1642.
[68]Nakhaeizadeh G, Schnabl A. Development of multi-criteria metrics for evaluation of data mining algorithms[C]. Third International Conference on Knowledge Discovery and Data Mining,1997,37-42.
[69]Challagulla Vub, Farokh Bb, Yen I-L, et al. Empirical assessment of machine learning based software defect prediction techniques[J]. International Journal on Artificial Intelligence Tools, 2008,17(2):389-400.
[70]Hartigan JA. Clustering algorithms[M]. John Wiley & Sons,1975.
[71]Jain AK, Dubes RC. Algorithms for clustering data[M]. Prentice-Hall,1988.
[72]Jain AK, Murty MN, Flynn PJ. Data clustering:a review[J]. ACM Computing Surveys,1999, 31(3):264-323.
[73]Hartigan JA, Wong MA. Algorithm AS 136:A k-means clustering algorithm[J]. Applied Statistics,1979:100-108.
[74]Birant D, Kut A. ST-DBSCAN:an algorithm for clustering spatial-temporal data[J]. Data & Knowledge Engineering,2007,60(1):208-221.
[75]Guha S, Rastogi R, Shim K. CURE:an efficient clustering algorithm for large databases[C]. ACM SIGMOD Record, ACM,1998,73-84.
[76]Livny TZRRM. BIRCH:an efficient data clustering method for very large databases[C]. ACM SIGMOD International Conference on Management of Data,1996,103-114.
[77]Liu M, Li F, Wu C. The order structure of fuzzy numbers based on the level characteristics and its application to optimization problems[J]. Science in China Series F:Information Sciences, 2002,45(6):433-441.
[78]Banfield JD, Raftery AE. Model-based Gaussian and non-Gaussian clustering[J]. Biometrics. 1993:803-821.
[79]McLachlan GJ, Basford KE. Mixture models:inference and applications to clustering[J]. Statistics:Textbooks and Monographs, New York:Dekker,1988,1.
[80]Steinbach M, Karypis G, Kumar V. A comparison of document clustering techniques[C]. KDD Workshop on Text Mining, Boston,2000,525-526.
[81]Fecker D, Margner V, Fingscheidt T. Training of classifiers for quality control of on-line laser brazing processes with highly imbalanced datasets[J]. Pattern Recognition,2012:367-376.
[82]Airman El, Avery RB, Eisenbeis RA, et al. Application of classification techniques in business, banking and finance[M]. JAI press,1981.
[83]Japkowicz N. The class imbalance problem:significance and strategies[C]. Proceedings of the 2000 International Conference on Artificial Intelligence,2000,1:111-117.
[84]Chawla NV, Japkowicz N, Kotcz A. Editorial:special issue on learning from imbalanced data sets[J]. ACM SIGKDD Explorations Newsletter,2004,6(1):1-6.
[85]Chawla NV, Bowyer KW, Hall LO, et al. SMOTE:synthetic minority over-sampling technique[J]. Journal of Artificial Intelligence Research,2002,16:321-357.
[86]Chawla NV. Data mining for imbalanced datasets:an overview[J]. Data Mining and Knowledge Discovery Handbook,2010:875-886.
[87]He H, Garcia EA. Learning from imbalanced data[J]. IEEE Transactions on Knowledge and Data Engineering,2009,21(9):1263-1284.
[88]Abbasnejad ME, Ramachandram D, Mandava R. A survey of the state of the art in learning the kernels[J]. Knowledge and Information Systems,2011,31 (2):193-221.
[89]吴克寿,曾志强.非平衡数据集分类研究[J].计算机技术与发展.2011,21(9)：39-42.
[90]马胜祥,马建庆,杨明.基于核函数拟合的非平衡数据分类方法[J].计算机应用与软件,2010,27(004)：177-179.
[91]李正欣,赵林度.基于SMOTEBoost的非均衡数据集SVM分类器[J].系统工程,2008,26(5)：116-119.
[92]岳超源,系统工程.决策理论与方法[M].北京：科学出版社,2003.
[93]王国华,梁樑.决策理论与方法[M].合肥：中国科学技术大学出版社,2006.
[94]徐玖平,李军.多目标决策的理论与方法[M].北京：清华大学出版社有限公司,2005.
[95]陈埏.决策分析[M].北京：科学出版杜.1987.
[96]高洪深.决策支持系统(DSS)理论·方法·案例[M].北京：清华大学出版社,2005.
[97]杨雷.群体决策理论与应用：群体决策中的个体偏好集结方法研究[M].北京：经济科学 出版社,2004.
[98]杨善林.智能决策方法与智能决策支持系统[M].北京：科学出版社,2005.
[99]吴祈宗,朱心想.几种区间数判断矩阵排序权向量计算方法的比较研究[J].北京工商大学学报：自然科学版,2002,20(4)：53-57.
[100]郭显光.多指标综合评价中权数的确定[J].数量经济技术经济研究,1989,11(6)：49-52.
[101]陆明生.多目标决策中的权系数[J].系统工程理论与实践,1986,6(4)：77-78.
[102]王宗军.多目标权系数赋值方法及其选择策略[J].系统工程与电子技术,1993,15(6)：35-41.
[103]吴云燕,华中生,查勇.AHP中群决策权重的确定与判断矩阵的合并[J].运筹与管理,2003,12(4)：16-21.
[104]梁樑,熊立,王国华.一种群决策中专家客观权重的确定方法[J].系统工程与电子技术,2005,27(4)：652-655.
[105]夏洪胜,盛昭瀚,徐南荣.多层多目标决策方法的综述[J].系统工程与电子技术,1992,7：27-32.
[106]顾基发.多目标决策问题[J],自然杂志,1980,2：14.
[107]Hwang CL, Yoon K. Multiple attribute decision making:methods and applications:a state-of-the-art survey[M]. Springer-Verlag New York,1981.
[108]Abo-Sinna MA, Amer AH. Extensions of TOPSIS for multi-objective large-scale nonlinear programming problems[J]. Applied Mathematics and Computation,2005,162(1):243-256.
[109]Opricovic S, Tzeng GH. Compromise solution by MCDM methods:a comparative analysis of VIKOR and TOPSIS[J]. European Journal of Operational Research,2004,156(2):445-455.
[110]张建胜,吴祈宗TOPSIS方法在Vague集中的应用[J].北京理工大学学报,2006,26(10)：937-940.
[111]周晓光,张强,胡望斌.基于Vague集的TOPSIS方法及其应用[J].系统工程理论方法应用,2005,14(6)：537-541.
[112]Brans JP. L'ingenierie de la decision:I'elaboration d'instruments d'aide a la decision[M]. Universite Laval, Faculte des sciences de I'administration,1982.
[113]Brans JP, Mareschal B. How to Decide with PROMETHEE[J]. ULB and VUB Brussels Universities Vrije Universiteit Brussel,1986,254-258.
[114]Brans JP, Mareschal B. PROMETHEE methods[J]. Multiple Criteria Decision Analysis:State of the Art Surveys,2005:163-186.
[115]邓乐斌,梅晓玲.信息不完全确定的PROMETHEE/TOPSIS方法及应用[J].长江大学学报(自科版),2007,4(4)：36-37.
[116]王建军,杨德礼.信息系统外包决策的AHP与PROMETHEE方法[J].管理学报,2006,3(3)：287-291.
[117]Chu MT, Shyu J, Tzeng GH, et al. Comparison among three analytical methods for knowledge communities group-decision analysis[J]. Expert Systems with Applications,2007,33(4): 1011-1024.
[118]Opricovic S, Tzeng GH. Multicriteria planning of post-earthquake sustainable seconstruction[J]. Computer-Aided Civil and Infrastructure Engineering,2002,17(3): 211-220.
[119]Opricovic S. Multicriteria optimization of civil engineering systems[J]. Faculty of Civil Engineering, Belgrade,1998,2(1):5-21.
[120]Saaty TL. What is the analytic hierarchy process?[M]. Springer,1988.
[121]华中生,吴云燕,徐晓燕.一种AHP判断矩阵一致性调整的新方法[J].系统工程与电子技术,2003,25(1)：38-40.
[122]黄德才,郑河荣.AHP方法中判断矩阵的标度扩展构造法[J].系统工程,2003,21(1)：105-109.
[123]王建,黄凤岗,景韶光.AHP中判断矩阵一致性调整方法研究[J].系统工程理论与实践,2005,25(8)：85-91.
[124]肖四汉,樊治平,王梦光.群决策中两类偏好信息-AHP判断矩阵和模糊偏好关系矩阵的一致化方法[J].系统工程学报,2002,17(1)：82-86.
[125]Cooper WW, Seiford LM, Zhu J. Data envelopment analysis:History, models, and interpretations[J]. Handbook on data envelopment analysis,2011:1-39.
[126]马占新.数据包络分析方法的研究进展[J].系统工程与电子技术,2002,24(3)：42-46.
[127]魏权龄.数据包络分析(DEA)[J].科学通报,2000,45(17)：1793-1808.
[128]魏权龄.数据包络分析[M].北京：科学出版社,2004.
[129]Roy B. Classement et choix en presence de points de vue multiples[J]. RAIRO-Operations Research-Recherche Operationnelle,1968,2(V1):57-75.
[130]Figueira J, Mousseau V, Roy B. ELECTRE methods[J]. Multiple Criteria Decision Analysis: State of the Art Surveys,2005:133-153.
[131]Milani A, Shanian A, El-Lahham C. Using different ELECTRE methods in strategic planning in the presence of human behavioral resistance[J]. Advances in Decision Sciences,2006,1-19.
[132]龚本刚,华中生,檀大水.一种语言评价信息不完全的多属性群决策方法[J].中国管理科学,2007,15(1)：88-93.
[133]王坚强.模糊多准则决策方法研究综述[J].控制与决策,2008,23(6)：601-607.
[134]徐泽水.基于语言信息的决策理论与方法[M].北京：科学出版社,2008.
[135]徐玖平,陈建中.群决策理论与方法及实现[M].北京：清华大学出版社,2009.
[136]邓聚龙.灰色系统基本方法(第一版)[M].湖北：华中工学院山版社,1987.
[137]吕锋,崔晓辉.多目标决策灰色关联投影法及其应用[J].系统工程理论与实践,2002, 22(1)：103-107.
[138]赵艳林,杨绿峰,吕海波,等.不确定信息条件下的灰色模式识别[J].控制与决策,2003,18(5)：593-596.
[139]Doyle JR, Green RH. Cross-evaluation in DEA:improving discrimination among DMUs[J]. Infor-Information Systems and Operational Research,1995,33(3):205.
[140]Green RH, Doyle JR, Cook WD. Preference voting and project ranking using DEA and cross-evaluation[J]. European Journal of Operational Research,1996,90(3):461-472.
[141]Liang L, Wu J, Cook WD, Zhu J. Alternative secondary goals in DEA cross-efficiency evaluation[J]. International Journal of Production Economics,2008,113(2):1025-1030.
[142]樊治平,肖四汉.基于OWA算子的不同形式偏好信息的群决策方法[J].控制与决策,2001,16(B11)：749-752.
[143]徐泽水.一种不确定型OWA算子及其在群决策中的应用[J].东南大学学报：自然科学版,2002,32(1)：147-150.
[144]Yager RR. Applications and extensions of OWA aggregations[J]. International Journal of Man-Machine Studies.1992,37(1):103-122.
[145]陈衍泰,陈国宏,李美娟.综合评价方法分类及研究进展[J].管理科学学报,2004,7(2)：69-79.
[146]杜栋,庞庆华.现代综合评价方法与案例精选[M].北京：清华大学出版社有限公司,2005.
[147]Ephraim Y, Malah D. Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator[J]. IEEE Transactions on Acoustics, Speech and Signal Processing,1984,32(6):1109-1121.
[148]Barron A, Rissanen J, Yu B. The minimum description length principle in coding and modeling[J]. IEEE Transactions on Information Theory,1998,44(6):2743-2760.
[149]Grunwald PD. The minimum description length principle[M]. The MIT Press,2007.
[150]李军.不平衡数据学习的研究[D].吉林：吉林大学,2011.
[151]Rendell L, Cho H. Empirical learning as a function of concept character[J]. Machine Learning, 1990,5(3):267-298.
[152]Aha DW. Generalizing from case studies:a case study[C]. Proceedings of the Ninth International Conference on Machine Learning,1992,1-10.
[153]Brodley CE. Addressing the selective superiority problem:automatic algorithm/model class selection[C]. Proceedings of the Tenth International Conference on Machine Learning Citeseer, 1993,17-24.
[154]Lindner G, Studer R. AST:support for algorithm selection with a CBR approach[M]. Principles of Data Mining and Knowledge Discovery, Springer,199,418-423.
[155]Smith-Miles KA. Cross-disciplinary perspectives on meta-learning for algorithm selection[J]. ACM Computing Surveys,2008,41(1):6-15.
[156]UCI Machine Learning Repository[DB]. http://archive.ics.uci.edu/ml/.
[157]Bensusan H, Giraud-Carrier C, Kennedy C. A higher-order approach to meta-learning[C]. Proceedings of the Work-in-Progress Track at the 10th International Conference on Inductive Logic Programming,2000,33-42.
[158]Seewald AK, Petrak J, Widmer G. Hybrid decision tree learners with alternative leaf classifiers: An empirical study[C]. Proceedings of the Fourteenth International Florida Artificial Intelligence Research Society Conference, AAAI Press,2001,407-411.
[159]Todorovski L, Blockeel H, Dzeroski S. Ranking with predictive clustering trees[M]. Springer, 2002.
[160]Ali S, Smith KA. On learning algorithm selection for classification[J]. Applied Soft Computing,2006,6(2):119-138.
[161]Ali S, Smith-Miles KA. A meta-learning approach to automatic kernel selection for support vector machines[J]. Neurocomputing,2006,70(1):173-186.
[162]Castiello C, Castellano G, Fanelli AM. Meta-data:characterization of input features for meta-learning[M]. Modeling Decisions for Artificial Intelligence:Springer,2005,457-468.
[163]Ler D, Koprinska 1, Chawla S. Utilising regression-based landmarkers within a meta-learning framework for algorithm selection[C].22nd Annual International Conference on Machine Learning, Bonn 2005,139-147.
[164]Peterson AH, Martinez T. Estimating the potential for combining learning models[C].22nd Annual International Conference on Machine Learning, Bonn 2005,68-75.
[165]Prodromidis A, Chan P, Stolfo S. Meta-learning in distributed data mining systems:issues and approaches[J]. Advances in Distributed and Parallel Knowledge Discovery,2000,3:81-118.
[166]Kopf C, Taylor C, Keller J. Meta-analysis:from data characterisation for meta-learning to meta-regression[C]. Proceedings of the PKDD-00 Workshop on Data Mining, Decision Support, Meta-Learning and ILP, Citeseer,2000,54-67.
[167]Kuba P, Brazdil P, Soares C, Woznica A. Exploiting sampling and meta-learning for parameter setting support vector machines[C]. Proceedings of the IBERAMIA,2002,217-225.
[168]Soares C, Brazdil PB, Kuba P. A meta-learning method to select the kernel width in support vector regression[J]. Machine Learning,2004,54(3):195-209.
[169]Arinze B, Kim S-L, Anandarajan M. Combining and selecting forecasting models using rule based induction[J]. Computers & Operations Research,1997,24(5):423-433.
[170]Wang X, Smith K, Hyndman R. Characteristic-based clustering for time series data[J]. Data Mining and Knowledge Discovery,2006,13(3):335-364.
[171]Kou G, Peng Y, Shi Y, et al. Discovering credit card holders' behavior by multiple criteria linear programming [J]. Annals of Operations Research,2005,135(1):261-274..
[172]Kwak W, Shi Y, Eldridge S W, et al. Bankruptcy prediction for Japanese firms:using Multiple Criteria Linear Programming data mining approach [J]. International Journal of Business Intelligence and Data Mining,2006,1 (4):401-416.
[173]Kwak W, Shi Y, Kou G. Bankruptcy prediction for Korean firms after the 1997 financial crisis: using a multiple criteria linear programming data mining approach [J]. Review of Quantitative Finance and Accounting,2012,38(4):441-453.
[174]Chapman M, Callis P, Jackson W. Metrics Data Program[OL], NASA IV and V Facility. http://mdp.ivv.nasa.gov/,2004
[175]Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C. DBSMOTE:density based synthetic minority over-sampling technique[J]. Applied Intelligence,2011,36(3):664-684.
[176]Chawla N, Lazarevic A, Hall L, et al. SMOTEBoost:improving prediction of the minority class in boosting[J]. Knowledge Discovery in Databases:PKDD 2003,2003,107-119.
[177]Ramentol E, Caballero Y, Bello R, et al. SMOTE-RSB:a hybrid preprocessing approach based on oversampling and undersampling for high imbalanced data-sets using SMOTE and rough sets theory[J]. Knowledge and Information Systems,2012,33(2):245-265.
[178]Chen X, Wasikowski M. Fast:a roc-based feature selection metric for small samples and imbalanced data classification problems[C]. Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining,2008,124-132.
[179]Kumar PG, Bell JBB. Using continuous feature selection metrics to suppress the class imbalance problem[J]. International Journal of Scientific & Engineering Research,2012, 3(3):1-38.
[180]Zheng Z, Wu X, Srihari R. Feature selection for text categorization on imbalanced data[J]. ACM SIGKDD Explorations Newsletter,2004,6(1):80-89.
[181]Clemencon S, Vayatis N. Nonparametric estimation of the precision-recall curve[C]. Proceedings of the 26th Annual International Conference on Machine Learning, ACM,2009, 185-192.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |