基于特征评价的模式识别算法研究
|
摘要
欧氏距离是模式识别算法中最常采用的相似性计算量度。在计算数据间相似度时,欧氏距离为所有特征赋予相同的重要性,但是这与实际情况不符。尤其当特征维数较高时,大量不相关特征会影响欧氏距离计算的准确性,从而影响模式识别算法的性能,产生维数灾问题。通常维数灾问题通过特征选取算法解决。但是特征选取算法仅适用于解决特征与类存在较高相关性或者完全不相关的情形。本文通过特征评价解决特征与类存在不同相关度的维数灾问题。
针对模糊C均值算法的维数灾,提出了基于函数CFuzziness的特征权重的学习算法。特征权重学习算法为每个特征赋予一个权重,区分其对聚类的贡献。合理的权重值使得相似的数据彼此更靠近,不相似的数据相互远离,此时的聚类结果好。通过梯度下降算法极小化函数CFuzziness就可以为每个特征赋予一个合适的权重值。权重应用于模糊C均值算法,得到加权模糊C均值算法。加权模糊C均值算法强调重要特征的作用,消减冗余特征的作用,从而得到较好的聚类结果。实验表明,加权模糊C均值算法的聚类结果优于模糊C均值算法的聚类结果。
针对最近邻分类器的维数灾,提出了两种特征子集划分算法,并构造相应的多分类器融合系统。首先,特征集合被划分成若干特征子集;然后,每个特征子集由一个子分类器分类识别;最终,多个子分类器的分类结果融合输出。特征子集所包含的维数降低,子分类器的维数灾减轻。选用合理的特征子集划分算法保证子分类器的正确性和多样性,融合多个子分类器的分类结果就能够得到更好的分类性能。
本文构造了基于遗传算法的特征子集划分算法和基于互信息的特征子集划分算法。遗传算法根据多分类器的融合正确率,采用全局搜索机制寻找最优的特征子集划分,属于Wrapper类的特征子集划分算法,该算法能够为子分类器选取最适合的特征子集。互信息根据特征与类的相关性,通过前向贪心搜索机制为子分类器选取相应的特征子集,属于Filter类的特征子集划分算法,该算法具有时间复杂度小的优点。
本文提出了一种模糊最近邻分类器,并采用它为子分类器。最近邻分类器仅能够给出数据所属的类别信息;而模糊最近邻分类器能够给出数据在每一类中的隶属度,更有效的反映输出结果。
多个子分类器的决策通过模糊积分融合得到最终分类结果。模糊积分是基于模糊测度的融合算法。模糊测度用于度量子分类器的重要性,重要性根据训练数据学习得到。与其他融合算法相比,模糊积分不仅考虑了子分类器的实际输出,而且考虑了子分类器的重要性,融合效果好。实验表明,基于遗传算法和基于互信息划分特征子集的两种模糊最近邻融合算法的分类性能均优于最近邻分类器的分类性能。
本文将上述3种算法应用于识别Corel图像库。Corel图像库中每幅图片分别通过颜色直方图,颜色一致向量,PWT和Hu矩提取得到4个特征文件,作为图像识别实验系统的输入数据。加权模糊C均值算法图像聚类的结果优于模糊C均值算法。图像分类采用基于遗传算法划分特征子集的模糊最近邻融合算法和基于互信息划分特征子集的模糊最近邻融合算法。两种融合算法分类图像的结果明显优于最近邻分类器。由于两种特征子集划分算法采用了不同的策略,融合算法的分类性能依赖于不同的数据库有所不同。
The Euclidean distance is the commonly used similarity measure in pattern recognition algorithm. It assumes that each feature plays the same role in pattern recognition algorithm, but it is not in practice. When the size of feature dimensionality is higher, the Euclidean distance may be dominated by some irrelevant features. Therefore, the performance of pattern recognition algorithm based on the Euclidean distance will be affected, which is called the curse of dimensionality. It can be lessened by feature selection. When the relevance between feature and class is either highly correlated or completely irrelevant, feature selection can perform best. In this study, feature evaluation is used to deal with the problem with different relevance between feature and class.
For the curse of dimensionality in fuzzy c mean, feature weight learning algorithm with respect to index CFuzziness is proposed. Feature weight learning algorithm assigns each feature an importance degree denoting the role in clustering. An appropriate feature weight leads to that the data within one class are more similar and the data in different classes are more separate. In this case, the performance of clustering is better. When index CFuzziness gets its minimum value through the gradient descent technique, the appropriate feature weights are learned. Fuzzy c mean incorporated with feature weight forms the weighted fuzzy c mean. Weighted fuzzy c mean algorithm emphasizes the roles of important features and lessens the roles of irrelevant features. Experimental results show that the weighted fuzzy c mean outperforms fuzzy c mean in clustering.
For the curse of dimensionality in nearest neighbor classifier, two multiple classifier systems are proposed based on different feature subset partition methods. Firstly, it decomposes the feature set into several feature subsets. Then each feature subset is classified by one component classifier. Finally, multiple decisions from each component classifiers are combined. Because the size of dimensionality in feature subset is low, the curse of dimensionality is lessened. If there is diversity and accuracy among component classifiers generated by feature subset partition method, multiple classifier system gets a better performance.
In this paper, GA and mutual information are used to partition feature subset. According to the multiple classifier system’s accuracy, GA automatically fulfils the feature subset partition by a global search strategy, which belongs to wrapper method. The wrapper method may select the feature subset suitable for each component classifier. Mutual information selects the salient feature subset according to the relevance between feature and class by a forward greedy search strategy, which belongs to filter method. The filter method may be computationally efficient.
In this paper, fuzzy nearest neighbor classifier is proposed, which is adopted as the component classifier. Nearest neighbor classifier outputs the class of data. While fuzzy nearest neighbor classifier outputs the membership degree of data belonging to each class.
Fuzzy integral is adopted to combine multiple decisions from each component classifier with respect to fuzzy measure. The importance degree for each feature subset is measured by fuzzy measure, where the importance degree is learned by training data. In comparison with other combination method, fuzzy integral not only considers the output of each component classifier but also considers the importance degree for each feature subset. Therefore, it outperforms other combination methods. Experimental results show that both multiple fuzzy nearest neighbor classifier systems based on feature subset by GA and mutual information can get better performance than nearest neighbor classifier in classification.
In this paper, three proposed methods are used to recognize Corel image database. Four datasets are retrieved from Corel image database by color histogram, color coherence vector, PWT and Hu moments respectively, which input the image recognition experimental system. Experimental results of image clustering show that weighted fuzzy c mean is superior to fuzzy c mean. Image classification adopts multiple fuzzy nearest neighbor classifier system based on feature subset by GA and multiple fuzzy nearest neighbor classifier system based on feature subset by mutual information. The experimental results show that both multiple fuzzy nearest neighbor classifier system improves the performance of image classification by nearest neighbor classifier. Because GA and mutual information adopt different strategies to partition feature subset, the performance of multiple classifier system depends on the dataset.
针对模糊C均值算法的维数灾,提出了基于函数CFuzziness的特征权重的学习算法。特征权重学习算法为每个特征赋予一个权重,区分其对聚类的贡献。合理的权重值使得相似的数据彼此更靠近,不相似的数据相互远离,此时的聚类结果好。通过梯度下降算法极小化函数CFuzziness就可以为每个特征赋予一个合适的权重值。权重应用于模糊C均值算法,得到加权模糊C均值算法。加权模糊C均值算法强调重要特征的作用,消减冗余特征的作用,从而得到较好的聚类结果。实验表明,加权模糊C均值算法的聚类结果优于模糊C均值算法的聚类结果。
针对最近邻分类器的维数灾,提出了两种特征子集划分算法,并构造相应的多分类器融合系统。首先,特征集合被划分成若干特征子集;然后,每个特征子集由一个子分类器分类识别;最终,多个子分类器的分类结果融合输出。特征子集所包含的维数降低,子分类器的维数灾减轻。选用合理的特征子集划分算法保证子分类器的正确性和多样性,融合多个子分类器的分类结果就能够得到更好的分类性能。
本文构造了基于遗传算法的特征子集划分算法和基于互信息的特征子集划分算法。遗传算法根据多分类器的融合正确率,采用全局搜索机制寻找最优的特征子集划分,属于Wrapper类的特征子集划分算法,该算法能够为子分类器选取最适合的特征子集。互信息根据特征与类的相关性,通过前向贪心搜索机制为子分类器选取相应的特征子集,属于Filter类的特征子集划分算法,该算法具有时间复杂度小的优点。
本文提出了一种模糊最近邻分类器,并采用它为子分类器。最近邻分类器仅能够给出数据所属的类别信息;而模糊最近邻分类器能够给出数据在每一类中的隶属度,更有效的反映输出结果。
多个子分类器的决策通过模糊积分融合得到最终分类结果。模糊积分是基于模糊测度的融合算法。模糊测度用于度量子分类器的重要性,重要性根据训练数据学习得到。与其他融合算法相比,模糊积分不仅考虑了子分类器的实际输出,而且考虑了子分类器的重要性,融合效果好。实验表明,基于遗传算法和基于互信息划分特征子集的两种模糊最近邻融合算法的分类性能均优于最近邻分类器的分类性能。
本文将上述3种算法应用于识别Corel图像库。Corel图像库中每幅图片分别通过颜色直方图,颜色一致向量,PWT和Hu矩提取得到4个特征文件,作为图像识别实验系统的输入数据。加权模糊C均值算法图像聚类的结果优于模糊C均值算法。图像分类采用基于遗传算法划分特征子集的模糊最近邻融合算法和基于互信息划分特征子集的模糊最近邻融合算法。两种融合算法分类图像的结果明显优于最近邻分类器。由于两种特征子集划分算法采用了不同的策略,融合算法的分类性能依赖于不同的数据库有所不同。
The Euclidean distance is the commonly used similarity measure in pattern recognition algorithm. It assumes that each feature plays the same role in pattern recognition algorithm, but it is not in practice. When the size of feature dimensionality is higher, the Euclidean distance may be dominated by some irrelevant features. Therefore, the performance of pattern recognition algorithm based on the Euclidean distance will be affected, which is called the curse of dimensionality. It can be lessened by feature selection. When the relevance between feature and class is either highly correlated or completely irrelevant, feature selection can perform best. In this study, feature evaluation is used to deal with the problem with different relevance between feature and class.
For the curse of dimensionality in fuzzy c mean, feature weight learning algorithm with respect to index CFuzziness is proposed. Feature weight learning algorithm assigns each feature an importance degree denoting the role in clustering. An appropriate feature weight leads to that the data within one class are more similar and the data in different classes are more separate. In this case, the performance of clustering is better. When index CFuzziness gets its minimum value through the gradient descent technique, the appropriate feature weights are learned. Fuzzy c mean incorporated with feature weight forms the weighted fuzzy c mean. Weighted fuzzy c mean algorithm emphasizes the roles of important features and lessens the roles of irrelevant features. Experimental results show that the weighted fuzzy c mean outperforms fuzzy c mean in clustering.
For the curse of dimensionality in nearest neighbor classifier, two multiple classifier systems are proposed based on different feature subset partition methods. Firstly, it decomposes the feature set into several feature subsets. Then each feature subset is classified by one component classifier. Finally, multiple decisions from each component classifiers are combined. Because the size of dimensionality in feature subset is low, the curse of dimensionality is lessened. If there is diversity and accuracy among component classifiers generated by feature subset partition method, multiple classifier system gets a better performance.
In this paper, GA and mutual information are used to partition feature subset. According to the multiple classifier system’s accuracy, GA automatically fulfils the feature subset partition by a global search strategy, which belongs to wrapper method. The wrapper method may select the feature subset suitable for each component classifier. Mutual information selects the salient feature subset according to the relevance between feature and class by a forward greedy search strategy, which belongs to filter method. The filter method may be computationally efficient.
In this paper, fuzzy nearest neighbor classifier is proposed, which is adopted as the component classifier. Nearest neighbor classifier outputs the class of data. While fuzzy nearest neighbor classifier outputs the membership degree of data belonging to each class.
Fuzzy integral is adopted to combine multiple decisions from each component classifier with respect to fuzzy measure. The importance degree for each feature subset is measured by fuzzy measure, where the importance degree is learned by training data. In comparison with other combination method, fuzzy integral not only considers the output of each component classifier but also considers the importance degree for each feature subset. Therefore, it outperforms other combination methods. Experimental results show that both multiple fuzzy nearest neighbor classifier systems based on feature subset by GA and mutual information can get better performance than nearest neighbor classifier in classification.
In this paper, three proposed methods are used to recognize Corel image database. Four datasets are retrieved from Corel image database by color histogram, color coherence vector, PWT and Hu moments respectively, which input the image recognition experimental system. Experimental results of image clustering show that weighted fuzzy c mean is superior to fuzzy c mean. Image classification adopts multiple fuzzy nearest neighbor classifier system based on feature subset by GA and multiple fuzzy nearest neighbor classifier system based on feature subset by mutual information. The experimental results show that both multiple fuzzy nearest neighbor classifier system improves the performance of image classification by nearest neighbor classifier. Because GA and mutual information adopt different strategies to partition feature subset, the performance of multiple classifier system depends on the dataset.
引文
1 孙继祥. 现代模式识别. 长沙: 国防科技大学出版社. 2002.
2 T.M. Mitchell. Machine learning. New York: The McGraw-Hill companies. 1997.
3 R.O. Duda, P.E. Hart, D.G. Stork. Pattern classification. NewYork: John Wiley & Sons INC. 2001.
4 边肇祺, 张学工. 模式识别. 北京: 清华大学出版社. 1999.
5 张智星, 孙春在, 水谷英二. 神经-模糊和软计算. 西安: 西安交通大学出版社. 2000.
6 A.K. Jain, R.C. Dubes. Algorithm for clustering data. New Jersey: Prentice-Hall, Englewood Cliffs. 1988.
7 J.C. Bezdek. Pattern recognition with fuzzy objective function algorithms. New York: Plenum Press. 1981.
8 B.V. Dasarathy. Nearest neighbor (NN) norms: NN pattern classification techniques. Los Alamitos: IEEE Computer Society Press. 1991.
9 F. Marcelloni. Feature selection based on a modified C-means algorithm with supervision. Information Sciences. 2003, 151:201-226.
10 R. Battiti. Using mutual information for selecting features in supervised neural net learning. IEEE Trans. on Neural Networks. 1994, 5(4):537-550.
11 A. Jain, D. Zongker. Feature selection: evaluation, application, and small sample performance. IEEE Trans. on PAMI. 1997, 19(2):153-158.
12 N. Kwan, C.H. Choi. Input feature selection by mutual information based on parzen window. IEEE Trans. on PAMI. 2002, 24(12):1667-1671.
13 S.K. Pal, P.K. De, J. Basak. Unsupervised feature evaluation: a neuro-fuzzy approach. IEEE Trans. on Neural Networks. 2000, 11(2):366-376.
14 R. Kohavi, G.H. John. Wrappers for feature subset selection. Artificial Intelligence. 1997, 97(1-2):273-324.
15 Z. X. Zhu, Y. S. Ong, M. Dash. Wrapper-Filter feature selection algorithm using a memetic framwork. IEEE Trans. on SMC part B. 2007, 37(1):70-76.
16 P. Mitra, C.A. Murthy, S.K. Pal. Unsupervised feature selection using featuresimilarity. IEEE Trans. on PAMI. 2002, 24(3):301-312.
17 W.H. Hsu. Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning. Information Sciences. 2004, 16(1-3):103-122.
18 N. Kwak, C.H. Choi. Input feature selection for classification problems. IEEE Trans. on NN. 2002, 13(1):143-159.
19 M. Dash, H. Liu. Feature selection for clustering. Proc. of 4th Pacific Asia conf. on Knowledge Discovery and Data Mining. Kyoto. 2000, 1508:110-121.
20 J. Dy, C. Brodley. Feature subset selection and order identification for unsupervised learning. Proc. of 17th conf. on Machine Learning. San Francisco. 2000:247-254.
21 S. Basu, C.A. Micchelli, P. Olsen. Maximum entropy and maximum likelihood criteria for feature selection from multivariate data. Proc. of IEEE int'l Sympsium on Circuits and Systems. Geneva. 2000:267-270.
22 M.A. Hall. Correlation based feature selection for discrete and numeric class machine learning. Proc. of 17th conf. on Machine Learning. San Francisco. 2000:359-366.
23 S.K. Das. Feature selection with a linear dependence measure. IEEE Trans. on Computers. 1971:1106-1109.
24 K. Kira, L. Rendell. A practical approach to feature selection. Proc. of 9th int'l workshop on Machine Learning. Florida. 1992:249-256.
25 D. Koller, M. Sahami. Towards optimal feature selection. Proc. of 13th intl'l conf. on Macnine Learning. Bari.1996:284-292.
26 P. Somol, P. Pudil, J. Kittler. Fast branch & bound algorithms for optimal feature selection. IEEE Trans. on PAMI. 2004, 26(7):900-912.
27 X.W. Chen. An improved branch and bound algorithm for feature selection. Pattern Recognition Letters. 2003, 24(12):1925-1933.
28 B. Yu, B.Z. Yuan. A more efficient branch and bound algorithm for feature selection. Pattern Recognition Letters. 1993, 26(6):883-889.
29 E. Krusinska. Two step semi-optimal branch and bound algorithm for feature selection in mixed variable discrimination. Pattern Recognition Letters. 1989, 22(4):455-459.
30 D.W. Aha, R.L. Bankert. A comparative evaluation of sequential feature selection algorithms. Proc. of the 5th International Workshop on Artificial Intelligence and Statistics. New York. 1996:1-7.
31 P.A. Devijver, J. Kittler. On the edited nearest neighbor rule. Proc. of the 5th Int’l conf. on Pattern Recognition. Florida. 1980:72-80.
32 P. Pudil, J. Novovi_cova, J. Kittler. Floating search methods in feature selection. Pattern Recognition Letters. 1994, 15(1):119-125.
33 M.A. Tahir, A. Bouridane, F. Kurugollu. Simultaneous feature selection and feature weighting using hybrid Tabu search/K-nearest neighbor classifier. Pattern Recognition Letters. 2007, 28(4):438-446.
34 H.B. Zhang, G.Y. Sun. Optimal reference subset selection for nearest neighbor classification by tabu search. Pattern Recognition. 2002, 35(7):1481-1490.
35 D. Wettschereck, D.W. Aha, T. Mohri. A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms. Artificial Intelligence Review. 1997, 11(1-5):273-314.
36 M.L. Raymer. Dimensionality reduction using Genetic Algorithms. IEEE Trans. on Evolution Computation. 2000, 4(2):164-171.
37 D. Lowe. Similarity metric learning for a variable kernel classifier. Neural Computation. 1995, 7(1):72-85.
38 X.Z. Wang, Y.D. Wang, L.J. Wang. Improving fuzzy c-means clustering based on feature-weight learning. Pattern Recognition Letters. 2004, 25(10):1123-1132.
39 张敏, 马少平, 宋睿华. DF还是IDF? 主特征模型在web信息检索中的使用. 软件学报. 2005, 16(05): 1012-1020.
40 赵珊, 崔江涛, 周利华. 基于位平面分布熵的图像检索算法. 电子与信息学报. 2007, 29(4): 795-799.
41 赵恒, 杨万海. 基于属性加权的模糊 K-Modes 聚类算法. 系统工程与电子技术. 2003, 25(10): 1299-1302.
42 陈振洲, 李磊, 姚正安. 基于 SVM 的特征加权 KNN 算法. 中山大学学报. 2005, 44(1): 17-20.
43 J. Kittler, M. Hatef, R.P.W. Duin. On combining classifiers. IEEE Trans. on PAMI. 1998, 20(3):226-239.
44 S. K. Pal, A. Pal. Pattern recognition: From classical to modern approaches. Singapore: World scientific. 2001:427-451.
45 A. Verikas, A. Lipnickas, K. Malmgvist, et al. Soft combination of neural classifiers: a comparative study. Pattern Recognition Letters. 1999, 20(4):429-444.
46 L.I. Kuncheva, C. Whitaker. Measures of diversity in classifier ensembles and their relationship with ensemble accuracy. Machine Learning. 2003, 51(2):181-207.
47 L. Bremain. Bagging predictors. Machine Learning. 1996, 24(2):123-140.
48 M. Skurichina, R.P.W. Duin. Bagging, boosting and the random subspace method for linear classifiers. Pattern Analysis and Application. 2002, 5(2):121-115.
49 Y. Freund. Boosting a weak learning algorithm by majority. Proc. of the 3rd annual workshop on computational learning theory. San Francisco. 1990: 202-216.
50 M.M. Islam, Y. Xin, K. Murase. A constructive algorithm for training cooperative neural network ensembles. IEEE Trans. on Neural Networks. 2003, 14(4):820-834.
51 L. Bremain. Random forest. Machine Learning. 2001, 45(1):5-32.
52 R. Bryll, R. Gutierrez-Osuna, F. Quek. Attribute bagging: Improving accuracy of classifier ensembles by using random features subsets. Pattern Recognition. 2003, 36(6):1291-1302.
53 L.I. Kuncheva, L.C. Jain. Designing classifier fusion systems by genetic algorithms. IEEE Trans. on Evolutionary Computation. 2000, 4(4):327-336.
54 P. Viswanath, M.N. Murty, C. Bhatnagar. Fusion of multiple approximate nearest neighbor classifier for fast and efficient classification. Information Fusion. 2004, 5(4):239-250.
55 A.P.F. Chan, W.W.Y. Ng, D.S. Yeung, et al. Multiple classifier system with feature grouping for intrusion detection: mutual information approach. Lecture Notes in Computer Science. 2005, 3683:141-148.
56 N.C. Oza, K. Tumer. Input decimation ensembles: decorrelation through dimensionality reduction. Lecture Notes in Computer Science. 2001, 2096:238-247.
57 P. Langley, W. Iba. Average-case analysis of a nearest neighbor algorithm. Proc. of 13th int'l conf. on Artificial Intelligence. Cambridge. 1993:889-894.
58 S.D. Bay. Nearest neighbor classification from multiple feature subsets. Intelligent Data Analysis. 1999, 3(3):191-209.
59 Z. Zhou, Y. Yu. Ensembling local learners through multimodal perturbation. IEEE Trans. on SMC part B. 2005, 35(4):725-735.
60 N. García-Pedrajas, D. Ortiz-Boyer. A cooperative constructive method for neural networks for pattern recognition. Pattern Recognition. 2007, 40(1):80-98.
61 T.K. Ho. The random subspace method for constructing decision forests. IEEE Trans. on PAMI. 1998, 20(8):832-844.
62 R. Maclin, J.W. Shavlik. Combining the predictions of multiple classifiers: using competitive learning to initialize neural networks. Proc. of the 14th int’l joint conf. on Artificial Intelligence. Montreal. 1995:524-530.
63 T.G. Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Machine Learning. 2000, 40(2):139-157.
64 L. Lam, C.Y. Suen. Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Trans. on SMC part A. 1997, 27(5):553 - 568.
65 A. Narasimhamurthy. Theoretical bounds of majority voting performance for a binary classification problem. IEEE Trans. on PAMI. 2005, 27(12):1988 -1995.
66 M. Taniguchi, V. Tresp. Averaging regularized estimators. Neural Computation. 1997, 9(5):1163-1178.
67 J.B.D. Cabrera. On the impact of fusion strategies on classification errors for large ensembles of classifiers. Pattern Recognition. 2006, 39(11):963-978.
68 D.M.J. Tax, M.V. Breukelen, R.P.W. Duin. Combining multiple classifiers by averaging or by multiplying. Pattern Recognition. 2000, 33:1475-1485.
69 T.K. Ho, J.J. Hull, S.N. Srihari. Decision combination in multiple classifier systems. IEEE Trans. on PAMI. 1993, 16(1):66 -75.
70 H. Zouari, L. Heutte, Y. Lecourtier. Experimental comparison of combinationrules using simulated data. Proc. of 18th Int’l conf. on Pattern Recognition. Hong Kong. 2006, 3:152-155.
71 L. Lam, C.Y. Suen. Optimal combination of pattern classifiers. Pattern Recognition Letters. 1995, 16(9):945-954.
72 V. Zorkadis, D.A. Karras, M. Panayotou. Efficient information theoretic strategies for classifier combination, feature extraction and performance evaluation in improving false positives and false negatives for spam e-mail filtering. Neural Networks. 2005, 18(5-6):799-807.
73 D. Heckerman. Bayes networks for data mining. Data mining and Knowledge Discovery. 1997, 1(1):79-119.
74 D. Harmanec, G.J. Klir, Z.Y. Wang. Modal logic interpretation of Dempster-Shafer theory: An infinite case. International Journal of Approximating Reasoning. 1996, 14(2-3):81-93.
75 L.I. Kuncheva. Switching between selection and fusion in combining classifiers: an experiment. IEEE Trans. on SMC Part B. 2002, 32(2):146 -156.
76 C. Dietrich, G. Palm, F. Schwenker. Decision templates for the classification of bioacoustic time series. Information Fusion. 2003, 4(2):101-109.
77 M. Sugeno. Fuzzy measures and fuzzy integrals: A survey. Fuzzy automata and decision process. Amsterdam: North Holland, 1977: 89-102.
78 Z.Y. Wang, G.J. Klir. Fuzzy measure theory. NewYork: Plenum Press; 1992.
79 L. Mikenina, H.J. Zimmermann. Improved feature selection and classification by the 2-additive fuzzy measure. Fuzzy Sets and Systems. 1999, 107:197-218.
80 石爱业, 徐立中. 一种自适应模糊密度赋值的小波变换遥感图像融合算法. 电子与信息学报. 2007, 29(2): 355-359.
81 Z.Y. Wang, K.S. Leung, M.L. Wong, J. Fang. A new type of nonlinear integrals and the computational algorithm. Fuzzy Sets and Systems. 2000, 112(2):223-231.
82 Z.Y. Wang, K.S. Leung, J. Wang. A genetic algorithm for determining nonadditive set functions in information fusion. Fuzzy Sets and Systems. 1999, 102(3):463-469.
83 Z.Y. Wang, K.B. Xu, J. Wang, G.J. Klir. Using genetic algorithms to determine nonnegative monotone set functions for information fusion in environmentswith random perturbation. International Journal of Intelligent Systems. 1999, 14(10):949-962.
84 J. Wang, Z.Y. Wang. Using neural networks to determine Sugeno measures by statics. Neural Networks. 1997, 10(1):183-195.
85 J.M. Keller, J. Osborn. Training the fuzzy integral. Journal of Approximate Reasoning. 1996, 15:1-24.
86 Z.Y. Wang, K.S. Leung, G.J. Klir. Applying fuzzy measures and nonlinear integrals in data mining. Fuzzy Sets and Systems. 2005, 156(3):371-380.
87 林剑, 王润生, 尹喜云. 维数差异对模糊积分融合多 FasART 网络影响的研究. 电子学报. 2006, 34(11):355-359.
88 A. Verikas, A. Lipnickas. Fusing neural network through space partitioning and fuzzy integration. Neural Processing Letters. 2002, 16(1):53-65.
89 J.C. Dunn. Some recent investigations of a new fuzzy partition algorithm and its application to pattern classification problems. Journal of Cybernetics. 1974, 4:1-15.
90 R. Sara?o?lu, K. Tütüncü, N. Allahverdi. A fuzzy clustering approach for finding similar documents using a novel similarity measure. Expert Systems with Applications. 2007, 33(3):600-605.
91 J. Lim, J. Park, G.G. Medioni. Text segmentation in color images using tensor voting. Image and Vision Computing. 2007, 25(5): 671-685.
92 R.R. Yager, D.P. Filev. Approximate clustering via the mountain method. IEEE Trans. on SMC part A. 1994, 24(8):1279-1284.
93 M. Ester, H.P. Kriegel, J. Sander, X. Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. Proc. of the Knowledge Discovery and Data Mining. 1996:226-231.
94 L.O. Hall, I.B. ?zyret, J.C. Bezdek. Clustering with a genetically optimized approach. IEEE Trans. on Evolutionary Computation. 1999, 3(2):103-111.
95 T.W. Cheng, D.B. Goldgof, L.O. Hall. Fast fuzzy clustering. Fuzzy Sets System. 1998, 93(1):49-56.
96 H.J. Sun, S.R. Wang, Q.S. Jiang. FCM based model selection algorithms for determining the number of clusters. Pattern Recognition. 2004, 37(10):2027-2037.
97 U. Kaymak, M. Setnes. Fuzzy clustering with volume prototypes and adaptive cluster merging. IEEE Trans. on Fuzzy Systems. 2002, 10(6):705-712.
98 D.E. Gustafson, W. Kessel. Fuzzy clustering with a fuzzy covariance matrix. Proc. of IEEE conf. on Decision Control. San Diego. 1979:761-766.
99 P.J. Rouseeuw, L. Kaufman, E. Trauwaert. Fuzzy clustering using scatter matrices. Computational Statistics Data Analysis. 1996, 23(1):135-151.
100 R. Krishnapuram, J. Kim. Clustering algorithms based on volume criteria. IEEE Trans. on Fuzzy Systems. 2000, 8(2):228-236.
101 W. Pedrycz, J. Waletzky. Fuzzy clutering with partial supervision. IEEE Trans. on SMC part B. 1997, 27(5):787-795.
102 K.L. Wu, J. Yu, M.-S. Yang. A novel fuzzy clustering algorithm based on a fuzzy scatter matrix with optimality tests. Pattern Recognition Letters. 2005, 26(5):639-652.
103 李洁, 高新波, 焦李成. 基于特征加权的模糊聚类新算法. 电子学报. 2006, 34(1): 89-92.
104 高新波, 李洁, 姬红兵. 基于加权模糊 c 均值聚类与统计检验指导的多阈值图像自动分割算法. 电子学报. 2004, 32(4): 661-664.
105 W. Pedrycz, G. Vukovich. Fuzzy clustering with supervision. Pattern Recognition. 2004, 37(7):1339-1349.
106 K. Hirota, W. Pedrycz. Directional fuzzy clustering and its application to fuzzy modeling. Fuzzy Sets and Systems. 1996, 80(3):315-326.
107 W. Pedrycz. Conditional fuzzy clustering in the design of radial basis function neural networks. IEEE Trans. on Neural Networks. 1998, 9(4):601-612.
108 A.M. Bensaid, L.O. Hall, J.C. Bezdek, et al. Validity-guided (re)clustering with applications to image segmentation. IEEE Trans. on Fuzzy System 1996, 4(2):112-123.
109 N.R. Pal, J.C. Bezdek. On cluster validity for the fuzzy c-mean model. IEEE Trans. on Fuzzy Systems. 1995, 3(3):370-379.
110 J.C. Bezdek. Cluster validity with fuzzy sets. Jouranl of Cybernet. 1974, 3(3):58-73.
111 J.C. Bezdek. Mathematical models for systematic and taxonomy. Proc. of 8th Int’l conf. Numerical Taxonomy. San Francisco. 1975:143-166.
112 M.P. Windham. Cluster validity for the fuzzy c-means clustering algorithm. IEEE Trans. on PAMI. 1982, 4:357-363.
113 R. Gunderson. Application of fuzzy ISODATA algorithms to star tracker pointing systems. Proc. of the 7th Triennial World IFCA Cong. Helsinki. 1978:1319-1323.
114 L.X. Xie, G. Beni. A validity measure for fuzzy clustering. IEEE Trans. on PAMI. 1991, 13(8):841-847.
115 Y. Fukuyama, T. Sugeno. A new method of choosing the number of clusters for the fuzzy c-means method. Proc. 5th Fuzzy Systems Symp. Japan. 1989:247-250.
116 J.C. Bezdek. A convergence theorem for the fuzzy ISODATA clustering algorithms. IEEE Trans. on PAMI. 1980, 2(1):1-8.
117 J.C. Bezdek, R.J. Hathaway, M.J. Sabin, et al. Convergence theory for fuzzy c-means: counterexamples and repairs. IEEE Trans. on SMC part B. 1987, 17(5):873-877.
118 T.M. Cover, P.E. Hart. Nearest neighbor pattern classification. IEEE Trans. on Information Theory. 1968, 13:21-27.
119 V. Ananthanarayana, M.N. Murty, D. Subramanian. An incremental data mining algorithm for compact realization of prototypes. Pattern Recognition. 2001, 34(11):2249-2251.
120 G. Grahne, J. Zhu. Fast algorithms for frequent itemset mining using FP-trees. IEEE Trans. on Knowledge and Data Engineering. 2005, 17(10):1347-1362.
121 Z. Tian, R. Raghu, L. Micon. BIRCH: an efficient data clustering methods for very large databases. Proc. of ACM SIGMOD int’l conf. of management data. Montreal. 1996:103-114.
122 A. Djouadi, E. Bouktache. A fast algorithm for the nearest-neighbor classifier. IEEE Trans. on PAMI. 1997, 19(3):277-281.
123 A. Guttman. R-trees: A dynamic index structure for spatial searching. Proc. of the ACM SIGMOD. Boston. 1984: 47-57.
124 Y. Hamaoto, S. Uchimra, S. Tomita. A bootstrap technique for nearest neighbor classifier design. IEEE Trans. on PAMI. 1997, 19(1):73-79.
125 P.E. Hart. The condensed nearest neighbor rule. IEEE Trans. on InformationTheory. 1968, 14(3):515-516.
126 G.W. Gates. The reduced nearest neighbor rule. IEEE Trans. on Information Theory. 1972, 18(3):431-433.
127 S. Watanabe. Frontiers of Pattern Recognition. New York: Academic Press. 1972: 511-519.
128 B.V. Dasarathy. Minimal consistent set (MCS) identification for optimal nearest neighbor decision systems design. IEEE Trans. on SMC part A. 1994, 24(3):511-517.
129 C.L. Chang. Finding prototypes for nearest neighbor classifiers. IEEE Trans. on Computers. 1974, 23(11):1179-1184.
130 M. Agrawal, N. Gupta, R. Shreelekshmi, M.N. Murty. Efficient pattern synthesis for nearest neighbor classifier. Pattern Recognition. 2005, 38(11):2200-2203.
131 S.Z. Li, J.W. Lu. Face recognition using the nearest feature line method. IEEE Trans. on Neural Networks. 1999, 10(2):439-443.
132 W.M. Zheng, L. Zhao, C.R. Zou. Locally nearest neighbor classifiers for pattern classification. Pattern Recognition. 2004, 37(6):1307-1309.
133 Q.B. Gao, Z.Z. Wang. Center-based nearest neighbor classifier. Pattern Recognition. 2007, 40(1):346-349.
134 Y.S. Huang, C.C. Chiang, J.W. Shieh. Prototype optimization for nearest-neighbor classification. Pattern Recognition. 2002, 35(6):1237-1245.
135 R.V.D. Heiden, F.C.A. Groen. The box-cox metric for nearest neighbor classification improvement. Pattern Recognition. 1997, 30(2):273-279.
136 C. Domeniconi, D. Gunopulos, J. Peng. Large margin nearest neighbor classifiers. IEEE Trans. on Neural Networks. 2005, 16(4):899-909.
137 J.H. Chen, H.M. Chen, S.Y. Ho. Design of nearest neighbor classifiers: multi-objective approach. International Journal of Approximate Reasoning. 2005, 40(1-2):3-22.
138 L.I. Kuncheva, L.C. Jain. Nearest neighbor classifier: Simultaneous editing and feature selection. Pattern Recognition Letters. 1999, 20(11-13):1149-1156.
139 E. Alpaydin. Voting over multiple condensed nearest neighbors. Artificial Intelligence Review. 1997, 11(1-5):115-132.
140 王熙照, 王丽娟等. 传递闭包聚类中的模糊性分析. 计算机工程与应用. 2003, 39(18):92-94.
141 A. DeLuca, S. Termini. A definition of a nonprobabilistic entropy in the setting of fuzzy set theory. Inform and Control. 1972, 20:301-312.
142 王丽娟. 特征权学习在无导师聚类中的应用. 河北大学硕士学位论文. 2003: 6-30.
143 刘纪平, 汪宏斌, 汪诚波, 周洞汝. 基于模糊最近邻的高维数据聚类. 小型微型计算机系统. 2005, 26(2):261-263.
144 S.X. Yu, S.D. Backer, P. Scheunders. Genetic feature selection combined with composite fuzzy nearest neighbor classifiers for hyperspectral satellite imagery. Pattern Recognition Letters. 2002, 23(1-3):183-190.
145 M. Sarkar. Fuzzy-rough nearest neighbor algorithms in classification. Fuzzy Sets and Systems. 2007, 158(19):2134 – 2152.
146 A.K. Jain, B. Chandrasekharan. Dimensionality and sample size considerations in pattern recognition practice. Handbook of Statistics. 1982, 2(39):835-855.
147 林磊. 多分类器融合技术的研究及其在汉字识别中的应用. 哈尔滨工业大学博士学位论文. 2004:13-26.
148 M. Grabish. Fuzzy integral for classification and feature extraction. Fuzzy Measures and Integrals Theory and Applications. Physica Verlag. 2000: 348-374.
149 T.W.S. Chow, D. Huang. Estimating optimal feature subsets using efficient estimation of high-dimensional mutual information. IEEE Trans. on NN. 2005, 16(1):213-224.
150 P.M. Narendra, K. Fukunaga. A branch and bound algorithm for feature selection. IEEE Trans. on Computer. 1977, C-26(9):917-922.
151 D. Franc-oisa, F. Rossib, V. Wertza, M. Verleysenc. Resampling methods for parameter-free and robust feature selection with mutual information. Neurocomputing. 2007, 70(7-9):1276-1288.
152 J. Huang, Y. Cai, X. Xu. A hybrid genetic algorithm for feature selection wrapper based on mutual information. Pattern Recognition Letters. 2007, 28(13): 1825-1844.
153 王亮申. 图像特征提取及基于内容图像数据库检索理论和方法研究. 大连理工大学博士学位论文. 2002: 1-65.
154 王艳妮, 陈龙斌, 王卫宏, 胡包钢. 一种基于语义的图像数据库分类系统. 计算机应用研究. 2004, 21(4): 256-260.
155 付岩, 王耀威, 王伟强, 高文. SVM 用于基于内容的自然图像分类和检索. 计算机学报. 2003, 26(10): 1261-1265.
156 M. A. Stricker, M. Orengo. Similarity of color image. Proc. of SPIE: Storage and Retrival for Image and Video Databases III. San Jose. 1995: 305-313.
157 M. F. A. Fauzi, P. H. Lewis. Texture based image retrieval using multiscale sub-image matching. Proc. of SPIE conf. on Image and Video Communications and Processing. California. 2003: 407-416.
158 G. Pass, R. Zabih. Histogram refinement for content-based image retrieval. Proc. of IEEE Workshop on Application of Computer Vision. Foorida. 1996: 96-102.
159 M. K. Hu. Visual pattern recognition by moment invariant. IRE Trans. on Information Theory. 1962, IT(8):179-187.
160 D. Kapur, Y.N. Lakshman, T. Saxena. Computing invariant using elimination methods. Proc. of int'l sysposium on Computer Vision. Florida. 1995: 97-102.
161 A. Vailaya, A. Jain, H.J. Zhang. On image classification: city image vs. landscapes. Pattern Recognition. 1998, 31 (12): 1921-1936.
162 D. Zhong, H.J. Zhang, S.F. Chang. Clustering methods for video browsing and annotation. Proc. of SPIE conf. on Storage and Retrieval for Image and Video database Ⅳ. San Jose. 1996: 239-246.
2 T.M. Mitchell. Machine learning. New York: The McGraw-Hill companies. 1997.
3 R.O. Duda, P.E. Hart, D.G. Stork. Pattern classification. NewYork: John Wiley & Sons INC. 2001.
4 边肇祺, 张学工. 模式识别. 北京: 清华大学出版社. 1999.
5 张智星, 孙春在, 水谷英二. 神经-模糊和软计算. 西安: 西安交通大学出版社. 2000.
6 A.K. Jain, R.C. Dubes. Algorithm for clustering data. New Jersey: Prentice-Hall, Englewood Cliffs. 1988.
7 J.C. Bezdek. Pattern recognition with fuzzy objective function algorithms. New York: Plenum Press. 1981.
8 B.V. Dasarathy. Nearest neighbor (NN) norms: NN pattern classification techniques. Los Alamitos: IEEE Computer Society Press. 1991.
9 F. Marcelloni. Feature selection based on a modified C-means algorithm with supervision. Information Sciences. 2003, 151:201-226.
10 R. Battiti. Using mutual information for selecting features in supervised neural net learning. IEEE Trans. on Neural Networks. 1994, 5(4):537-550.
11 A. Jain, D. Zongker. Feature selection: evaluation, application, and small sample performance. IEEE Trans. on PAMI. 1997, 19(2):153-158.
12 N. Kwan, C.H. Choi. Input feature selection by mutual information based on parzen window. IEEE Trans. on PAMI. 2002, 24(12):1667-1671.
13 S.K. Pal, P.K. De, J. Basak. Unsupervised feature evaluation: a neuro-fuzzy approach. IEEE Trans. on Neural Networks. 2000, 11(2):366-376.
14 R. Kohavi, G.H. John. Wrappers for feature subset selection. Artificial Intelligence. 1997, 97(1-2):273-324.
15 Z. X. Zhu, Y. S. Ong, M. Dash. Wrapper-Filter feature selection algorithm using a memetic framwork. IEEE Trans. on SMC part B. 2007, 37(1):70-76.
16 P. Mitra, C.A. Murthy, S.K. Pal. Unsupervised feature selection using featuresimilarity. IEEE Trans. on PAMI. 2002, 24(3):301-312.
17 W.H. Hsu. Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning. Information Sciences. 2004, 16(1-3):103-122.
18 N. Kwak, C.H. Choi. Input feature selection for classification problems. IEEE Trans. on NN. 2002, 13(1):143-159.
19 M. Dash, H. Liu. Feature selection for clustering. Proc. of 4th Pacific Asia conf. on Knowledge Discovery and Data Mining. Kyoto. 2000, 1508:110-121.
20 J. Dy, C. Brodley. Feature subset selection and order identification for unsupervised learning. Proc. of 17th conf. on Machine Learning. San Francisco. 2000:247-254.
21 S. Basu, C.A. Micchelli, P. Olsen. Maximum entropy and maximum likelihood criteria for feature selection from multivariate data. Proc. of IEEE int'l Sympsium on Circuits and Systems. Geneva. 2000:267-270.
22 M.A. Hall. Correlation based feature selection for discrete and numeric class machine learning. Proc. of 17th conf. on Machine Learning. San Francisco. 2000:359-366.
23 S.K. Das. Feature selection with a linear dependence measure. IEEE Trans. on Computers. 1971:1106-1109.
24 K. Kira, L. Rendell. A practical approach to feature selection. Proc. of 9th int'l workshop on Machine Learning. Florida. 1992:249-256.
25 D. Koller, M. Sahami. Towards optimal feature selection. Proc. of 13th intl'l conf. on Macnine Learning. Bari.1996:284-292.
26 P. Somol, P. Pudil, J. Kittler. Fast branch & bound algorithms for optimal feature selection. IEEE Trans. on PAMI. 2004, 26(7):900-912.
27 X.W. Chen. An improved branch and bound algorithm for feature selection. Pattern Recognition Letters. 2003, 24(12):1925-1933.
28 B. Yu, B.Z. Yuan. A more efficient branch and bound algorithm for feature selection. Pattern Recognition Letters. 1993, 26(6):883-889.
29 E. Krusinska. Two step semi-optimal branch and bound algorithm for feature selection in mixed variable discrimination. Pattern Recognition Letters. 1989, 22(4):455-459.
30 D.W. Aha, R.L. Bankert. A comparative evaluation of sequential feature selection algorithms. Proc. of the 5th International Workshop on Artificial Intelligence and Statistics. New York. 1996:1-7.
31 P.A. Devijver, J. Kittler. On the edited nearest neighbor rule. Proc. of the 5th Int’l conf. on Pattern Recognition. Florida. 1980:72-80.
32 P. Pudil, J. Novovi_cova, J. Kittler. Floating search methods in feature selection. Pattern Recognition Letters. 1994, 15(1):119-125.
33 M.A. Tahir, A. Bouridane, F. Kurugollu. Simultaneous feature selection and feature weighting using hybrid Tabu search/K-nearest neighbor classifier. Pattern Recognition Letters. 2007, 28(4):438-446.
34 H.B. Zhang, G.Y. Sun. Optimal reference subset selection for nearest neighbor classification by tabu search. Pattern Recognition. 2002, 35(7):1481-1490.
35 D. Wettschereck, D.W. Aha, T. Mohri. A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms. Artificial Intelligence Review. 1997, 11(1-5):273-314.
36 M.L. Raymer. Dimensionality reduction using Genetic Algorithms. IEEE Trans. on Evolution Computation. 2000, 4(2):164-171.
37 D. Lowe. Similarity metric learning for a variable kernel classifier. Neural Computation. 1995, 7(1):72-85.
38 X.Z. Wang, Y.D. Wang, L.J. Wang. Improving fuzzy c-means clustering based on feature-weight learning. Pattern Recognition Letters. 2004, 25(10):1123-1132.
39 张敏, 马少平, 宋睿华. DF还是IDF? 主特征模型在web信息检索中的使用. 软件学报. 2005, 16(05): 1012-1020.
40 赵珊, 崔江涛, 周利华. 基于位平面分布熵的图像检索算法. 电子与信息学报. 2007, 29(4): 795-799.
41 赵恒, 杨万海. 基于属性加权的模糊 K-Modes 聚类算法. 系统工程与电子技术. 2003, 25(10): 1299-1302.
42 陈振洲, 李磊, 姚正安. 基于 SVM 的特征加权 KNN 算法. 中山大学学报. 2005, 44(1): 17-20.
43 J. Kittler, M. Hatef, R.P.W. Duin. On combining classifiers. IEEE Trans. on PAMI. 1998, 20(3):226-239.
44 S. K. Pal, A. Pal. Pattern recognition: From classical to modern approaches. Singapore: World scientific. 2001:427-451.
45 A. Verikas, A. Lipnickas, K. Malmgvist, et al. Soft combination of neural classifiers: a comparative study. Pattern Recognition Letters. 1999, 20(4):429-444.
46 L.I. Kuncheva, C. Whitaker. Measures of diversity in classifier ensembles and their relationship with ensemble accuracy. Machine Learning. 2003, 51(2):181-207.
47 L. Bremain. Bagging predictors. Machine Learning. 1996, 24(2):123-140.
48 M. Skurichina, R.P.W. Duin. Bagging, boosting and the random subspace method for linear classifiers. Pattern Analysis and Application. 2002, 5(2):121-115.
49 Y. Freund. Boosting a weak learning algorithm by majority. Proc. of the 3rd annual workshop on computational learning theory. San Francisco. 1990: 202-216.
50 M.M. Islam, Y. Xin, K. Murase. A constructive algorithm for training cooperative neural network ensembles. IEEE Trans. on Neural Networks. 2003, 14(4):820-834.
51 L. Bremain. Random forest. Machine Learning. 2001, 45(1):5-32.
52 R. Bryll, R. Gutierrez-Osuna, F. Quek. Attribute bagging: Improving accuracy of classifier ensembles by using random features subsets. Pattern Recognition. 2003, 36(6):1291-1302.
53 L.I. Kuncheva, L.C. Jain. Designing classifier fusion systems by genetic algorithms. IEEE Trans. on Evolutionary Computation. 2000, 4(4):327-336.
54 P. Viswanath, M.N. Murty, C. Bhatnagar. Fusion of multiple approximate nearest neighbor classifier for fast and efficient classification. Information Fusion. 2004, 5(4):239-250.
55 A.P.F. Chan, W.W.Y. Ng, D.S. Yeung, et al. Multiple classifier system with feature grouping for intrusion detection: mutual information approach. Lecture Notes in Computer Science. 2005, 3683:141-148.
56 N.C. Oza, K. Tumer. Input decimation ensembles: decorrelation through dimensionality reduction. Lecture Notes in Computer Science. 2001, 2096:238-247.
57 P. Langley, W. Iba. Average-case analysis of a nearest neighbor algorithm. Proc. of 13th int'l conf. on Artificial Intelligence. Cambridge. 1993:889-894.
58 S.D. Bay. Nearest neighbor classification from multiple feature subsets. Intelligent Data Analysis. 1999, 3(3):191-209.
59 Z. Zhou, Y. Yu. Ensembling local learners through multimodal perturbation. IEEE Trans. on SMC part B. 2005, 35(4):725-735.
60 N. García-Pedrajas, D. Ortiz-Boyer. A cooperative constructive method for neural networks for pattern recognition. Pattern Recognition. 2007, 40(1):80-98.
61 T.K. Ho. The random subspace method for constructing decision forests. IEEE Trans. on PAMI. 1998, 20(8):832-844.
62 R. Maclin, J.W. Shavlik. Combining the predictions of multiple classifiers: using competitive learning to initialize neural networks. Proc. of the 14th int’l joint conf. on Artificial Intelligence. Montreal. 1995:524-530.
63 T.G. Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Machine Learning. 2000, 40(2):139-157.
64 L. Lam, C.Y. Suen. Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Trans. on SMC part A. 1997, 27(5):553 - 568.
65 A. Narasimhamurthy. Theoretical bounds of majority voting performance for a binary classification problem. IEEE Trans. on PAMI. 2005, 27(12):1988 -1995.
66 M. Taniguchi, V. Tresp. Averaging regularized estimators. Neural Computation. 1997, 9(5):1163-1178.
67 J.B.D. Cabrera. On the impact of fusion strategies on classification errors for large ensembles of classifiers. Pattern Recognition. 2006, 39(11):963-978.
68 D.M.J. Tax, M.V. Breukelen, R.P.W. Duin. Combining multiple classifiers by averaging or by multiplying. Pattern Recognition. 2000, 33:1475-1485.
69 T.K. Ho, J.J. Hull, S.N. Srihari. Decision combination in multiple classifier systems. IEEE Trans. on PAMI. 1993, 16(1):66 -75.
70 H. Zouari, L. Heutte, Y. Lecourtier. Experimental comparison of combinationrules using simulated data. Proc. of 18th Int’l conf. on Pattern Recognition. Hong Kong. 2006, 3:152-155.
71 L. Lam, C.Y. Suen. Optimal combination of pattern classifiers. Pattern Recognition Letters. 1995, 16(9):945-954.
72 V. Zorkadis, D.A. Karras, M. Panayotou. Efficient information theoretic strategies for classifier combination, feature extraction and performance evaluation in improving false positives and false negatives for spam e-mail filtering. Neural Networks. 2005, 18(5-6):799-807.
73 D. Heckerman. Bayes networks for data mining. Data mining and Knowledge Discovery. 1997, 1(1):79-119.
74 D. Harmanec, G.J. Klir, Z.Y. Wang. Modal logic interpretation of Dempster-Shafer theory: An infinite case. International Journal of Approximating Reasoning. 1996, 14(2-3):81-93.
75 L.I. Kuncheva. Switching between selection and fusion in combining classifiers: an experiment. IEEE Trans. on SMC Part B. 2002, 32(2):146 -156.
76 C. Dietrich, G. Palm, F. Schwenker. Decision templates for the classification of bioacoustic time series. Information Fusion. 2003, 4(2):101-109.
77 M. Sugeno. Fuzzy measures and fuzzy integrals: A survey. Fuzzy automata and decision process. Amsterdam: North Holland, 1977: 89-102.
78 Z.Y. Wang, G.J. Klir. Fuzzy measure theory. NewYork: Plenum Press; 1992.
79 L. Mikenina, H.J. Zimmermann. Improved feature selection and classification by the 2-additive fuzzy measure. Fuzzy Sets and Systems. 1999, 107:197-218.
80 石爱业, 徐立中. 一种自适应模糊密度赋值的小波变换遥感图像融合算法. 电子与信息学报. 2007, 29(2): 355-359.
81 Z.Y. Wang, K.S. Leung, M.L. Wong, J. Fang. A new type of nonlinear integrals and the computational algorithm. Fuzzy Sets and Systems. 2000, 112(2):223-231.
82 Z.Y. Wang, K.S. Leung, J. Wang. A genetic algorithm for determining nonadditive set functions in information fusion. Fuzzy Sets and Systems. 1999, 102(3):463-469.
83 Z.Y. Wang, K.B. Xu, J. Wang, G.J. Klir. Using genetic algorithms to determine nonnegative monotone set functions for information fusion in environmentswith random perturbation. International Journal of Intelligent Systems. 1999, 14(10):949-962.
84 J. Wang, Z.Y. Wang. Using neural networks to determine Sugeno measures by statics. Neural Networks. 1997, 10(1):183-195.
85 J.M. Keller, J. Osborn. Training the fuzzy integral. Journal of Approximate Reasoning. 1996, 15:1-24.
86 Z.Y. Wang, K.S. Leung, G.J. Klir. Applying fuzzy measures and nonlinear integrals in data mining. Fuzzy Sets and Systems. 2005, 156(3):371-380.
87 林剑, 王润生, 尹喜云. 维数差异对模糊积分融合多 FasART 网络影响的研究. 电子学报. 2006, 34(11):355-359.
88 A. Verikas, A. Lipnickas. Fusing neural network through space partitioning and fuzzy integration. Neural Processing Letters. 2002, 16(1):53-65.
89 J.C. Dunn. Some recent investigations of a new fuzzy partition algorithm and its application to pattern classification problems. Journal of Cybernetics. 1974, 4:1-15.
90 R. Sara?o?lu, K. Tütüncü, N. Allahverdi. A fuzzy clustering approach for finding similar documents using a novel similarity measure. Expert Systems with Applications. 2007, 33(3):600-605.
91 J. Lim, J. Park, G.G. Medioni. Text segmentation in color images using tensor voting. Image and Vision Computing. 2007, 25(5): 671-685.
92 R.R. Yager, D.P. Filev. Approximate clustering via the mountain method. IEEE Trans. on SMC part A. 1994, 24(8):1279-1284.
93 M. Ester, H.P. Kriegel, J. Sander, X. Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. Proc. of the Knowledge Discovery and Data Mining. 1996:226-231.
94 L.O. Hall, I.B. ?zyret, J.C. Bezdek. Clustering with a genetically optimized approach. IEEE Trans. on Evolutionary Computation. 1999, 3(2):103-111.
95 T.W. Cheng, D.B. Goldgof, L.O. Hall. Fast fuzzy clustering. Fuzzy Sets System. 1998, 93(1):49-56.
96 H.J. Sun, S.R. Wang, Q.S. Jiang. FCM based model selection algorithms for determining the number of clusters. Pattern Recognition. 2004, 37(10):2027-2037.
97 U. Kaymak, M. Setnes. Fuzzy clustering with volume prototypes and adaptive cluster merging. IEEE Trans. on Fuzzy Systems. 2002, 10(6):705-712.
98 D.E. Gustafson, W. Kessel. Fuzzy clustering with a fuzzy covariance matrix. Proc. of IEEE conf. on Decision Control. San Diego. 1979:761-766.
99 P.J. Rouseeuw, L. Kaufman, E. Trauwaert. Fuzzy clustering using scatter matrices. Computational Statistics Data Analysis. 1996, 23(1):135-151.
100 R. Krishnapuram, J. Kim. Clustering algorithms based on volume criteria. IEEE Trans. on Fuzzy Systems. 2000, 8(2):228-236.
101 W. Pedrycz, J. Waletzky. Fuzzy clutering with partial supervision. IEEE Trans. on SMC part B. 1997, 27(5):787-795.
102 K.L. Wu, J. Yu, M.-S. Yang. A novel fuzzy clustering algorithm based on a fuzzy scatter matrix with optimality tests. Pattern Recognition Letters. 2005, 26(5):639-652.
103 李洁, 高新波, 焦李成. 基于特征加权的模糊聚类新算法. 电子学报. 2006, 34(1): 89-92.
104 高新波, 李洁, 姬红兵. 基于加权模糊 c 均值聚类与统计检验指导的多阈值图像自动分割算法. 电子学报. 2004, 32(4): 661-664.
105 W. Pedrycz, G. Vukovich. Fuzzy clustering with supervision. Pattern Recognition. 2004, 37(7):1339-1349.
106 K. Hirota, W. Pedrycz. Directional fuzzy clustering and its application to fuzzy modeling. Fuzzy Sets and Systems. 1996, 80(3):315-326.
107 W. Pedrycz. Conditional fuzzy clustering in the design of radial basis function neural networks. IEEE Trans. on Neural Networks. 1998, 9(4):601-612.
108 A.M. Bensaid, L.O. Hall, J.C. Bezdek, et al. Validity-guided (re)clustering with applications to image segmentation. IEEE Trans. on Fuzzy System 1996, 4(2):112-123.
109 N.R. Pal, J.C. Bezdek. On cluster validity for the fuzzy c-mean model. IEEE Trans. on Fuzzy Systems. 1995, 3(3):370-379.
110 J.C. Bezdek. Cluster validity with fuzzy sets. Jouranl of Cybernet. 1974, 3(3):58-73.
111 J.C. Bezdek. Mathematical models for systematic and taxonomy. Proc. of 8th Int’l conf. Numerical Taxonomy. San Francisco. 1975:143-166.
112 M.P. Windham. Cluster validity for the fuzzy c-means clustering algorithm. IEEE Trans. on PAMI. 1982, 4:357-363.
113 R. Gunderson. Application of fuzzy ISODATA algorithms to star tracker pointing systems. Proc. of the 7th Triennial World IFCA Cong. Helsinki. 1978:1319-1323.
114 L.X. Xie, G. Beni. A validity measure for fuzzy clustering. IEEE Trans. on PAMI. 1991, 13(8):841-847.
115 Y. Fukuyama, T. Sugeno. A new method of choosing the number of clusters for the fuzzy c-means method. Proc. 5th Fuzzy Systems Symp. Japan. 1989:247-250.
116 J.C. Bezdek. A convergence theorem for the fuzzy ISODATA clustering algorithms. IEEE Trans. on PAMI. 1980, 2(1):1-8.
117 J.C. Bezdek, R.J. Hathaway, M.J. Sabin, et al. Convergence theory for fuzzy c-means: counterexamples and repairs. IEEE Trans. on SMC part B. 1987, 17(5):873-877.
118 T.M. Cover, P.E. Hart. Nearest neighbor pattern classification. IEEE Trans. on Information Theory. 1968, 13:21-27.
119 V. Ananthanarayana, M.N. Murty, D. Subramanian. An incremental data mining algorithm for compact realization of prototypes. Pattern Recognition. 2001, 34(11):2249-2251.
120 G. Grahne, J. Zhu. Fast algorithms for frequent itemset mining using FP-trees. IEEE Trans. on Knowledge and Data Engineering. 2005, 17(10):1347-1362.
121 Z. Tian, R. Raghu, L. Micon. BIRCH: an efficient data clustering methods for very large databases. Proc. of ACM SIGMOD int’l conf. of management data. Montreal. 1996:103-114.
122 A. Djouadi, E. Bouktache. A fast algorithm for the nearest-neighbor classifier. IEEE Trans. on PAMI. 1997, 19(3):277-281.
123 A. Guttman. R-trees: A dynamic index structure for spatial searching. Proc. of the ACM SIGMOD. Boston. 1984: 47-57.
124 Y. Hamaoto, S. Uchimra, S. Tomita. A bootstrap technique for nearest neighbor classifier design. IEEE Trans. on PAMI. 1997, 19(1):73-79.
125 P.E. Hart. The condensed nearest neighbor rule. IEEE Trans. on InformationTheory. 1968, 14(3):515-516.
126 G.W. Gates. The reduced nearest neighbor rule. IEEE Trans. on Information Theory. 1972, 18(3):431-433.
127 S. Watanabe. Frontiers of Pattern Recognition. New York: Academic Press. 1972: 511-519.
128 B.V. Dasarathy. Minimal consistent set (MCS) identification for optimal nearest neighbor decision systems design. IEEE Trans. on SMC part A. 1994, 24(3):511-517.
129 C.L. Chang. Finding prototypes for nearest neighbor classifiers. IEEE Trans. on Computers. 1974, 23(11):1179-1184.
130 M. Agrawal, N. Gupta, R. Shreelekshmi, M.N. Murty. Efficient pattern synthesis for nearest neighbor classifier. Pattern Recognition. 2005, 38(11):2200-2203.
131 S.Z. Li, J.W. Lu. Face recognition using the nearest feature line method. IEEE Trans. on Neural Networks. 1999, 10(2):439-443.
132 W.M. Zheng, L. Zhao, C.R. Zou. Locally nearest neighbor classifiers for pattern classification. Pattern Recognition. 2004, 37(6):1307-1309.
133 Q.B. Gao, Z.Z. Wang. Center-based nearest neighbor classifier. Pattern Recognition. 2007, 40(1):346-349.
134 Y.S. Huang, C.C. Chiang, J.W. Shieh. Prototype optimization for nearest-neighbor classification. Pattern Recognition. 2002, 35(6):1237-1245.
135 R.V.D. Heiden, F.C.A. Groen. The box-cox metric for nearest neighbor classification improvement. Pattern Recognition. 1997, 30(2):273-279.
136 C. Domeniconi, D. Gunopulos, J. Peng. Large margin nearest neighbor classifiers. IEEE Trans. on Neural Networks. 2005, 16(4):899-909.
137 J.H. Chen, H.M. Chen, S.Y. Ho. Design of nearest neighbor classifiers: multi-objective approach. International Journal of Approximate Reasoning. 2005, 40(1-2):3-22.
138 L.I. Kuncheva, L.C. Jain. Nearest neighbor classifier: Simultaneous editing and feature selection. Pattern Recognition Letters. 1999, 20(11-13):1149-1156.
139 E. Alpaydin. Voting over multiple condensed nearest neighbors. Artificial Intelligence Review. 1997, 11(1-5):115-132.
140 王熙照, 王丽娟等. 传递闭包聚类中的模糊性分析. 计算机工程与应用. 2003, 39(18):92-94.
141 A. DeLuca, S. Termini. A definition of a nonprobabilistic entropy in the setting of fuzzy set theory. Inform and Control. 1972, 20:301-312.
142 王丽娟. 特征权学习在无导师聚类中的应用. 河北大学硕士学位论文. 2003: 6-30.
143 刘纪平, 汪宏斌, 汪诚波, 周洞汝. 基于模糊最近邻的高维数据聚类. 小型微型计算机系统. 2005, 26(2):261-263.
144 S.X. Yu, S.D. Backer, P. Scheunders. Genetic feature selection combined with composite fuzzy nearest neighbor classifiers for hyperspectral satellite imagery. Pattern Recognition Letters. 2002, 23(1-3):183-190.
145 M. Sarkar. Fuzzy-rough nearest neighbor algorithms in classification. Fuzzy Sets and Systems. 2007, 158(19):2134 – 2152.
146 A.K. Jain, B. Chandrasekharan. Dimensionality and sample size considerations in pattern recognition practice. Handbook of Statistics. 1982, 2(39):835-855.
147 林磊. 多分类器融合技术的研究及其在汉字识别中的应用. 哈尔滨工业大学博士学位论文. 2004:13-26.
148 M. Grabish. Fuzzy integral for classification and feature extraction. Fuzzy Measures and Integrals Theory and Applications. Physica Verlag. 2000: 348-374.
149 T.W.S. Chow, D. Huang. Estimating optimal feature subsets using efficient estimation of high-dimensional mutual information. IEEE Trans. on NN. 2005, 16(1):213-224.
150 P.M. Narendra, K. Fukunaga. A branch and bound algorithm for feature selection. IEEE Trans. on Computer. 1977, C-26(9):917-922.
151 D. Franc-oisa, F. Rossib, V. Wertza, M. Verleysenc. Resampling methods for parameter-free and robust feature selection with mutual information. Neurocomputing. 2007, 70(7-9):1276-1288.
152 J. Huang, Y. Cai, X. Xu. A hybrid genetic algorithm for feature selection wrapper based on mutual information. Pattern Recognition Letters. 2007, 28(13): 1825-1844.
153 王亮申. 图像特征提取及基于内容图像数据库检索理论和方法研究. 大连理工大学博士学位论文. 2002: 1-65.
154 王艳妮, 陈龙斌, 王卫宏, 胡包钢. 一种基于语义的图像数据库分类系统. 计算机应用研究. 2004, 21(4): 256-260.
155 付岩, 王耀威, 王伟强, 高文. SVM 用于基于内容的自然图像分类和检索. 计算机学报. 2003, 26(10): 1261-1265.
156 M. A. Stricker, M. Orengo. Similarity of color image. Proc. of SPIE: Storage and Retrival for Image and Video Databases III. San Jose. 1995: 305-313.
157 M. F. A. Fauzi, P. H. Lewis. Texture based image retrieval using multiscale sub-image matching. Proc. of SPIE conf. on Image and Video Communications and Processing. California. 2003: 407-416.
158 G. Pass, R. Zabih. Histogram refinement for content-based image retrieval. Proc. of IEEE Workshop on Application of Computer Vision. Foorida. 1996: 96-102.
159 M. K. Hu. Visual pattern recognition by moment invariant. IRE Trans. on Information Theory. 1962, IT(8):179-187.
160 D. Kapur, Y.N. Lakshman, T. Saxena. Computing invariant using elimination methods. Proc. of int'l sysposium on Computer Vision. Florida. 1995: 97-102.
161 A. Vailaya, A. Jain, H.J. Zhang. On image classification: city image vs. landscapes. Pattern Recognition. 1998, 31 (12): 1921-1936.
162 D. Zhong, H.J. Zhang, S.F. Chang. Clustering methods for video browsing and annotation. Proc. of SPIE conf. on Storage and Retrieval for Image and Video database Ⅳ. San Jose. 1996: 239-246.