基于支持向量机的脱机手写相似汉字识别的研究
|
摘要
虽然脱机手写体汉字识别技术具有广阔的应用前景,但是由于脱机手写体汉字自身所特有的复杂性,使得识别系统的实现具有很大的难度,目前还没有十分成熟的产品。研究表明相似汉字的存在是影响系统识别率低的主要原因之一,因此我们必须花大力气解决手写体相似汉字的识别问题。鉴于支持向量机在小规模细分类问题上的优势,本文以脱机手写体相似汉字为对象,深入研究了基于支持向量机的手写体汉字识别中的若干核心问题,做了以下几方面具有创新性的工作：
首先,基于核函数的黎曼几何分析,提出了一种SVM自动模型选择方法。该方法先利用基于粗网格与模式搜索相结合的全局优化搜索算法,依据分类器性能评价准则来获得优化的SVM模型参数；之后再采用文中所提出的新保角变换,对核函数进行数据依赖性改进,进一步提高分类器泛化能力。
其次,研究了两种不同形式下的特征选择方案：①针对单目标特征选择问题,提出了一种基于单目标改进GA算法与交叉验证SVM分类的特征选择方案；②针对多目标特征选择问题,提出了一种基于Pareto优势的MOGA算法与SVM分类的特征选择方案。这两种方案均属于利用SVM分类器反馈信息的Wrapper求解方法,能在不降低系统泛化性能的情况下,获得维数较小的特征向量。
然后,针对DAGSVM分类器的存在问题,提出了一种新的基于结构优化的模糊多值DAGSVM分类器。根据分类器性能评价准则,给出了训练阶段离线获得结构优化DAGSVM的算法；在识别阶段,模糊多值DAGSVM分类器利用模糊隶属度函数与平均算子获得分类识别结果。与其它基于组合策略的多值SVM分类器相比,该分类器具有更高的识别精度和识别速度。
最后,在分析客观相似汉字的相似特性基础上,建立了一个较为实用的手写体相似汉字样本库,为今后的进一步研究奠定了基础；提出了一种基于小波弹性网格提取特征、利用遗传算法选择特征和SVM分类相结合的手写体相似汉字识别方案,实验结果表明了该识别方案的可行性和有效性。
Off-line handwritten Chinese characters recognition technology can be widely used inmany areas, but it is very difficult in realizing the recognition system because of its complexityand until now there is not perfect product. Investigation shows that the existence of lots ofsimilar characters is a key factor affecting recognition rate. Therefore, much deeper researchesshould be made to solve the difficult problem of similar handwritten Chinese characterrecognition. Since the approach based on support vector machines (SVM) is fit for smallsamples and small categories recognition problem, the dissertation investigates some keyproblems of SVM using in similar handwritten Chinese character recognition. The contributionsof this dissertation are presented as follows.
Firstly, an approach for automatic model selection for SVM classifier is proposed based onthe Riemannian geometry analysis of kernel function. The optimal model parameters areobtained based on the evaluation criterion of model selection and the global search algorithm ofcoarse grid combined with pattern search. Then, the novel conformal transformation presentedin this dissertation is adopted and the kernel function is modified by the transformation in adata-dependent way. And the experimental results show remarkable improvement of thegeneralization performance of the classifier.
Secondly, two different schemes for feature selection are investigated. One is based onsingle objective improved genetic algorithms and cross-validation-based SVM classification forsingle objective feature selection problem. The other is based on pareto-optimality-based multi-objecitve genetic algorithms and SVM classification for multi-objective feature selectionproblem. They all belong to the wrapper methods, which utilize the feedback information fromSVM classifier. The feature vector with low dimension can be found without any loss to thegeneralization performance.
Thirdly, aiming at the deficiency of DAGSVM classifier, a novel fuzzy multi-classDAGSVM classifier based on optimal structure is proposed. According to the performance evaluation criterion, the algorithm for optimizing the structure of DAGSVM is introduced in thetraining stage. The final recognition result is obtained by fuzzy multi-class DAGSVM classifierusing the fuzzy membership function and average operator in the testing stage. Theexperimental results show that the recognition precision and rate of the novel classifier are allbetter than those of pair-wise SVM classifier with other combination strategies.
Lastly, a practical sample library for similar handwritten Chinese characters is built on theanalysis of the similar property of objective similar handwritten Chinese characters, whichprovides the foundation for the future research. A recognition approach for similar handwrittenChinese characters is presented based on extracting feature vector by wavelet transformationand elastic meshing technique, selecting feature subset by genetic algorithms and classifying bySVM. The experimental results confirm the effectiveness and practicality of the approach.
首先,基于核函数的黎曼几何分析,提出了一种SVM自动模型选择方法。该方法先利用基于粗网格与模式搜索相结合的全局优化搜索算法,依据分类器性能评价准则来获得优化的SVM模型参数；之后再采用文中所提出的新保角变换,对核函数进行数据依赖性改进,进一步提高分类器泛化能力。
其次,研究了两种不同形式下的特征选择方案：①针对单目标特征选择问题,提出了一种基于单目标改进GA算法与交叉验证SVM分类的特征选择方案；②针对多目标特征选择问题,提出了一种基于Pareto优势的MOGA算法与SVM分类的特征选择方案。这两种方案均属于利用SVM分类器反馈信息的Wrapper求解方法,能在不降低系统泛化性能的情况下,获得维数较小的特征向量。
然后,针对DAGSVM分类器的存在问题,提出了一种新的基于结构优化的模糊多值DAGSVM分类器。根据分类器性能评价准则,给出了训练阶段离线获得结构优化DAGSVM的算法；在识别阶段,模糊多值DAGSVM分类器利用模糊隶属度函数与平均算子获得分类识别结果。与其它基于组合策略的多值SVM分类器相比,该分类器具有更高的识别精度和识别速度。
最后,在分析客观相似汉字的相似特性基础上,建立了一个较为实用的手写体相似汉字样本库,为今后的进一步研究奠定了基础；提出了一种基于小波弹性网格提取特征、利用遗传算法选择特征和SVM分类相结合的手写体相似汉字识别方案,实验结果表明了该识别方案的可行性和有效性。
Off-line handwritten Chinese characters recognition technology can be widely used inmany areas, but it is very difficult in realizing the recognition system because of its complexityand until now there is not perfect product. Investigation shows that the existence of lots ofsimilar characters is a key factor affecting recognition rate. Therefore, much deeper researchesshould be made to solve the difficult problem of similar handwritten Chinese characterrecognition. Since the approach based on support vector machines (SVM) is fit for smallsamples and small categories recognition problem, the dissertation investigates some keyproblems of SVM using in similar handwritten Chinese character recognition. The contributionsof this dissertation are presented as follows.
Firstly, an approach for automatic model selection for SVM classifier is proposed based onthe Riemannian geometry analysis of kernel function. The optimal model parameters areobtained based on the evaluation criterion of model selection and the global search algorithm ofcoarse grid combined with pattern search. Then, the novel conformal transformation presentedin this dissertation is adopted and the kernel function is modified by the transformation in adata-dependent way. And the experimental results show remarkable improvement of thegeneralization performance of the classifier.
Secondly, two different schemes for feature selection are investigated. One is based onsingle objective improved genetic algorithms and cross-validation-based SVM classification forsingle objective feature selection problem. The other is based on pareto-optimality-based multi-objecitve genetic algorithms and SVM classification for multi-objective feature selectionproblem. They all belong to the wrapper methods, which utilize the feedback information fromSVM classifier. The feature vector with low dimension can be found without any loss to thegeneralization performance.
Thirdly, aiming at the deficiency of DAGSVM classifier, a novel fuzzy multi-classDAGSVM classifier based on optimal structure is proposed. According to the performance evaluation criterion, the algorithm for optimizing the structure of DAGSVM is introduced in thetraining stage. The final recognition result is obtained by fuzzy multi-class DAGSVM classifierusing the fuzzy membership function and average operator in the testing stage. Theexperimental results show that the recognition precision and rate of the novel classifier are allbetter than those of pair-wise SVM classifier with other combination strategies.
Lastly, a practical sample library for similar handwritten Chinese characters is built on theanalysis of the similar property of objective similar handwritten Chinese characters, whichprovides the foundation for the future research. A recognition approach for similar handwrittenChinese characters is presented based on extracting feature vector by wavelet transformationand elastic meshing technique, selecting feature subset by genetic algorithms and classifying bySVM. The experimental results confirm the effectiveness and practicality of the approach.
引文
[1]边肇祺,张学工.模式识别.北京：清华大学出版社,2000.
[2]Vapnik V N. The nature of statistical learning theory. Springer-Verlag, London, UK,1995.
[3]Vapnik V N. Statistical learning theory. John Wiley&Sons, New York, USA,1998.
[4]Vapnik V N. The nature of statistical learning theory (Second Edition). Springer, New York, USA,1999.
[5]Tettamanzi A, Tomassim M. Soft computing: integrating evolutionary, neural, and fuzzy systems. Springer-Verlag, Berlin, Heidelberg,2001.
[6]Casey R, Nagy G. Recognition of printed Chinese character. IEEE Transaction on Electric Computer,1996,1(15):91-101.
[7]张忻中.我国汉字识别技术的历史、现状和展望.中文信息学报,1995,9(1)：25-30.
[8]赵宇明,江兴智,施鹏飞.基于笔划提取和合并的离线手写体汉字字符切分算法.红外与激光工程,2002,31(1)：23-27.
[9]周嫔,马少平,姜哲.基于单元合并的汉字切分算法的改进.中文信息学报,1999,13(2)：33-39.
[10]吕岳,施鹏飞,张克华.基于组件合并的手写体汉字串分割.软件学报,2000,11(11)：1154-1159.
[11]Zhao S Y, Chi Z R, Shi P F, Yan H. Two-stage segmentation unconstrained handwritten Chinese characters. Pattern Recognition,2003,36(1):145-156.
[12]朱小燕,王松.傅立叶变换在粘连文字图像切分中的应用.计算机学报,1999,22(12)：1246-1252.
[13]Tseng Y H, Lee H J. Recognition-based handwritten Chinese character segmentation using a probabilistic Viterbi algorithm. Pattern Recognition Letters,1999,20(8):791-806.
[14]赵斌,苏辉,夏绍玮.一种无约束手写体数字串分割方法.中文信息学报,1998,12(3)：21-28.
[15]Yamada H, Yamamoto K, Saito T. A nonlinear normalization method for handprinted Kanji character recognition: line density equalization. Pattern Recognition,1990,23(9):1023-1029.
[16]金连文,黄建成,尹俊勋,贺前华.变形变换及其在手写体汉字整形中的应用.中国图象图形学报,2002,7A(2)：170-175.
[17]王庆,赵荣椿,冯大淦.手写体汉字的规范化处理及评价.数据采集与处理,2001,16(2)：227-232.
[18]张睿,丁晓青,方驰.脱机手写汉字识别的最优采样特征新方法.中国图象图形学报,2002,7A(2)：176-180.
[19]马少平,夏莹,朱小燕.基于模糊方向线素特征的手写体汉字识别.清华大学学报(自然科学版),1997,37(3)：42-45.
[20]Lee H M, Huag C W, Sheu C C A fuzzy rule-based for handwritten Chinese characters recognition based on radical extraction. Fuzzy Sets and Systems,1998,100(1-3):59-70.
[21]陈津颖,金奕江,马少平.手写体汉字在特征空间的可视化分析.中文信息学报,2000,14(5)：42-48.
[22]张青,尹俊勋.小波变换在手写体汉字识别中的应用.电路与系统学报,1996,1(3)：63-67.
[23]Yu T, Ernest C M L, Tang Y Y. Feature extraction using wavelet and fractal. Pattern Recognition Letters,2001,22(3-4):271-287.
[24]金连文,彭秀兰,尹俊勋.一种手写体汉字特征提取新方法：小波变换及弹性网格技术的应用.中国图象图形学报,1998,3(7)：549-552.
[25]居琰,汪同庆,彭建.特征融合用于手写体汉字识别研究.电于科技大学学报,2002,31(3)：229-233.
[26]杨健,杨静宇,王正群,郭丽.一种组合特征抽取的新方法.计算机学报,2002,25(6)：570-575.
[27]王正群,孙兴华,郭丽,杨静宇.基于Fisher准则的多特征融合.计算机工程,2002,28(3)：41-42.
[28]金连文,梁宇杰.一种新的距离分类方法及其应用.计算机工程,1999,25(8)：30-31,57.
[29]盛立东,何其明.基于神经网络的手写体汉字识别实验系统.中文信息学报,1993,7(1)：48-57.
[30]邵秀丽,李勇建,蔡文进,刘憬.基干进化神经网络的手写体汉字识别.南开大学学报(自然科学版),2001,34(4)：53-56.
[31]吴鸣锐,张钹.一种用于大规模模式识别问题的神经网络算法.软件学报,2001,12(6)：851-855.
[32]金连文,徐秉铮RBF-DDA神经网络用于手写体汉字识别的研究.华南理工大学学报(自然科学版),1997,25(9)：97-101.
[33]Suganthan P N, Yan H. Recognition of handprinted Chinese characters by constrained graph matching. Image Vision Computing,1998,16(3):191-201.
[34]Chiu H P, Tseng D C. Invariant handwritten Chinese character recognition using fuzzy min-max neural networks. Pattern Recognition Letters,1997,18(5):481-491.
[35]金连文,徐秉铮.基于多级神经网络结构的手写体汉字识别.通信学报,1997,18(5)：21-27.
[36]韩宏,杨静宇.神经网络分类器的组合.计算机研究与发展,2000,37(12)：1488-1492.
[37]Devil H F Y, William W H Y. Different approach to designing neural network for similar handwritten Chinese character recognition. In: Proceedings of SPIE, Conference on Applications and Science of Neural Networks, Fuzzy Systems, and Evolutionary Computation,1998,3455:286-291.
[38]Liu C L, Kim I J, Kim J H. Model-based stroke extraction and matching for handwritten Chinese character recognition. Pattern Recognition,2001,34(12):2339-2352.
[39]Hsieh A J, Fan K C, Fan T I. Handwritten Chinese characters recognition by greedy matching with geometric constraint. Image Vision Computing,1996,14(2):91-104.
[40]高学,金连文,尹俊勋,黄建成.一种基于支持向量机的手写汉字识别方法.电子学报,2002,30(5)：651-654.
[41]范劲松,方廷健.基于粗集理论和SVM算法的模式分类方法.模式识别与人工智能,2000,13(4)：419-423.
[42]吕岳,施鹏飞,赵宇明.多分类器组合的投票表决规则.上海交通大学学报,2000,34(5)：680-683.
[43]孙怀江,胡钟山,杨静宇.基于证据理论的多分类器融合方法研究.计算机学报,2001,24(3)：231-235.
[44]征荆,丁晓青,吴佑寿.基于最小代价的多分类器动态集成.计算机学报,1999,22(2)：182-187.
[45]王春恒,肖柏华,戴汝为.基于串/并结合机制的集成型脱机手写汉字识别方法.模式识别与人工智能,1999,12(3)：255-261.
[46]韩宏,杨静宇,娄震.基于层次的分类器组合.南京理工大学学报,2002,26(1)：10-14.
[47]郭宏,丁晓青,郭繁夏,吴佑寿.汉字识别多分类器集成的新方法.清华大学学报(自然科学版),1997,37(10)：91-94.
[48]肖旭红,戴汝为.一种识别手写汉字的多分类器集成方法.自动化学报,1997,23(5)：621-627.
[49]蔡樱,盛立东.手写文稿识别的一种后处理方法和系统集成.中文信息学报,2000,14(3)：30-36.
[50]Chang C H. Simulated annealing clustering of Chinese words for contextual text recognition. Pattern Recognition Letters,1996,17(1):57-66.
[51]李元祥,丁晓青,刘长松.基于HMM的汉语文本识别后处理研究.中文信息学报,1999,13(4)：29-34.
[52]李元祥,丁晓青,吴佑寿.一种基于字词结合的汉字识别上下文处理新方法.计算机研究与发展,2002,39(7)：838-842.
[53]薛炳如,杨静宇,娄震,胡钟山.小类别数手写汉字建模.中文信息学报,1999,13(6)：34-40.
[54]石繁槐,童学锋.SVM在小字符集脱机手写体汉字识别中的应用研究计算机工程,2002,28(6)：187-189.
[55]朱小燕,史一凡.基于反馈的手写体字符识别方法的研究.计算机学报,2002,25(5)：476-482.
[56]姜珊,孙玉方.手写汉字的集群识别.中文信息学报,1998,12(4)：1-8.
[57]Cortes C, Vapnik V. Support vector networks. Machine Learning,1995,20(3):273-297.
[58]Osuna E, Freund R, Girosi F. An improved training algorithm for support vector machines. In: Proceedings of the1997IEEE Workshop on Neural Networks for Signal Processing VII,1997:276-285.
[59]Joachims T. Making large-scale SVM learning practical. In: Advances in Kernel Methods-Support Vector Learning, MIT Press, Cambridge, MA,1999:169-184.
[60]Platt J C. Fast training of support vetor machines using sequential minimal optimization. In: Advances in Kernel Methods-Support Vector Learning, MIT Press, Cambridge, MA,1999:185-208.
[61]Wahba G. Reproducing kernel Hilbert spaces-two brief reviews. Technical Report1079, Department of Statistics, University of Wisconsin-Madison, April2003:1-12.
[62]Platt J C, Cristianini N, Shawe-Taylor J. Large margin DAGs for multiclass classification. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA,2000,12:547-553.
[63]Dietterich T G, Bakiri G. Solving multi-class learning problems via error-correcting output codes. Journal of Artificial Intelligent Reasearch,1995,2:263-286.
[64]Hsu W, Lin C J. A comparision of methods for multiclass support vector machines. IEEE Transaction on Neural Networks,2002,13(2):415-425.
[65]Chapelle O, Vapnik V. Choosing multiple parameters for support vector machines. Machine Learning,2002,46(1-3):131-159.
[66]Cristianini N, Campbel C. Dynamically adapting kernels in support vector machines. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA,1999,11:204-210.
[67]Anguita D, Ridella S, Rivieccio F, Zunino R Hyperparameter design criteria for support vector classifers. Neurocomputing,2003,55:109-134.
[68]Duan K, Keerthi S S, Poo A N. Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing,2003,51:41-59.
[69]Gold C, Sollichb P. Model selection for support vector machine classifcation. Neurocomputing,2003,55:221-249.
[70]Chapelle O, Vapnik V N. Model selection for support vector machines. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA,2000,12:230-236.
[71]Ayat N E, Cheriet M, Remaki L, Suen C Y. KMOD-a new support vector machine kernel with moderate decreasing for pattern recognition. In: Proceedings of the6th International Conference on Document Analysis and Recognition,2001:1215-1219.
[72]张莉,周伟达,焦李成.一类新的支撑矢量机核.软件学报,2002,13(4)：713-718.
[73]Haasdonk B, Keysers D. Tangent distance kernels for support vector machines. In: Proceedings of the16th International Conference on Pattern Recognition,2002,2:864-868.
[74]张小云,刘允才.高斯核支撑向量机的性能分析.计算机工程,2003,29(8)：22-25.
[75]Wang W J, Xu Z B, Lu W Z, Zhang X Y. Determination of the spread parameter in the Gaussian kernel for classifcation and regression. Neurocomputing,2003,55:643-663.
[76]陈维桓,李兴校.黎曼几何引论.北京：北京大学出版社,2002.
[77]Amari S, Nagaoka H. Methods of information geometry. American Mathematical Society, USA,2000.
[78]Amari S, Wu S. Improving support vector machine classifiers by modifying kernel functions. Neural Networks,1999,12(6):783-789.
[79]Wu S, Amari S. Conformal transformation of kernel functions:a data-dependent way to improve support vector machine classifiers. Neural Processing Letters,2002,15(1):59-67.
[80]Luntz A, Brailovsky V. On estimation of characters obtained in statistical procedure of recognition. Technicheskaya Kibernetica,1969,3:1-20.
[81]Wahba G. Support vector machine, reproducing kernel Hilbert spaces and the randomize GACV. In: Advances in Kernel Methods-Support Vector Learning, MIT Press,1999:69-88.
[82]Wahba G, Lin Y, Zhang H. Generalize approximate cross validation for support vector machines. In: Advances in Large Margin Classifiers, MIT Press, Cambridge, MA,2000:297-311.
[83]Jaakola T S, Haussler D. Exploiting generative models in discriminative classifiers. In: Proceedings of the10th Conference on Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA,11,1999:487-493.
[84]邓乃扬,田英杰.数据挖掘中的新方法一支持向量机.北京：科学出版社,2004.
[85]周伟达,张莉,焦李成.一种改进的推广能力衡量准则.计算机学报,2003,26(5)：598-604.
[86]Joachims T. Estimating the generalizaition performance of an SVM efficientily. In: Proceedings of the17th International Conference on Machine Learning, San Franscisco, California,2000:431-438.
[87]Opper M, Winther O. Gaussian processes and svm: mean field and leave-one-out. In: Advances in Large Margin Classifiers, MIT Press, Cambridge, MA,2000:311-326.
[88]Momma M, Bennett K P. A pattern search method for model selection of support vector regression. In: Proceedings of the2th SIAM International Conference on Data Mining, Arlington, Virginia, April2002:261-274.
[89]Burges C J C. Geometry and invariance in kernel based methods. In: Advances in Kernel Methods-Support Vector Learning, Cambridge, MIT Press, Cambridge, MA,1999:89-116.
[90]周伟达,张莉,焦李成.支撑矢量机推广能力分析.电子学报,2001,29(5)：590-594.
[91]Barzilay O, Brailovsky V L. On domain knowledge and feature selection using a support vector machine. Pattern Recognition Letters,1999,20(5):475-484.
[92]Cao L J, Tay F E. Feature selection for support vector machines in financial time series forecasting. In: Proceedings of the2th International Conference on Intelligent Data Engineering and Automated Learning: Data Mining, Financial Engineering, and Intelligent Agents, Springer-Verlag, December, LNCS1983,2000:268-273.
[93]Kohavi R, John G H. Wrappers for feature subset selection. Artificial Intelligence,1997,(1-2):273-324.
[94]Zhang H, Sun G. Feature selection using tabu search method. Pattern Recognition,2002,35(3):701-711.
[95]Yamany S M, Khiani K J, Farag A A. Application of neural networks and genetic algorithms in the classification of endothelial cells. Pattern Recognition Letters,1997,18(11-13):1205-1210.
[96]Purshouse R C, Fleming P J. The multi-objective genetic algorithm applied to benchmark problems-an analysis. Research Report No.796, UK: University of Sheffield, Department of Automatic Control and Systems Engineering,2001,8:1-44.
[97]Horn J, Nafpliotis N, Goldberg D E. A niched pareto genetic algorithm for multiobjective optimization. In: Proceedings of the1th IEEE Conference on Evolutionary Computation, IEEE Press, Piscataway, NJ,1994:82-87.
[98]Goldberg D E. Genetic algorithms in search, optimization and machine learning. Reissue, Addison-Wesley Publishing Company,1989.
[99]Emmanouilidis C, Hunter A, Maclntyre J. A multiobjective evolutionary setting for feature selection and a commonality-based crossover operator. In: Proceedings of Congress on Evolutionary Computation,2000,1:309-316.
[100]Oliveira L S, Sabourin R, Bortolozzi F, Suen C Y. Feature selection using multi-objective genetic algorithms for handwritten digit recognition. In: Proceedings of16th International Conference on Pattern Recognition (ICPR),2002,1:568-571.
[101]Srinivas N, Deb K. Multiobjective optimization using nondominated sorting in genetic algorithms. Evolutionary Computation,1995,2(3):221-248.
[102]Lin C F, Wang S D. Fuzzy support vector machines. IEEE Transaction on Neural Networks,2002,13(3):464-471.
[103]Lin C F, Wang S D. Training algorithms for fuzzy support vector machines with noisy data. Pattern Recognition Letters,2004,25(14):1647-1656.
[104]Huang H P, Liu Y H. Fuzzy support vector machines for pattern recognition and data mining. International Journal of Fuzzy Systems,2002,4(3):826-835.
[105]Camps-Valls G, Martin-Guerrero J D, Rojo-Alvarez J L et al. Fuzzy sigmoid kernel for support vector classifiers. Neurocomputing,2004,62:501-506.
[106]Abe S. Analysis of multiclass support vector machines. In: Proceedings of International Conference on Computational Intelligence for Modeling Control and Automation, Vienna, Austria, Feburary2003:385-396.
[107]Inoue T, Abe S. Fuzzy support vector machines for pattern recognition. In: Proceedings of International Joint Conference on Neural Network, Washington, D.C., July2001,2:1449-1454.
[108]Tsang E C C, Yeung D S, Chan P P K. Fuzzy support vector machines for solving two-class problems. In: Proceedings of the2th International Conference on Machine Learning and Cybernetics, Xi'an, China, November2003,2:1080-1083.
[109]李昆仑,黄厚宽,田盛丰.模糊多类SVM模型.电子学报,2004,32(5)：830-832.
[110]Weston J, Watkins C. Support vector machines for multiclass pattern recognition. In: Proceedings of the7th European Symposium on Artificial Neural Networks, April1999:219-224.
[111]Kreβel U H-G. Pairwise classification and support vector machines. In: Advances in Kernel Methods:Support Vector Learning, MIT Press, Cambridge, MA,1999:255-268.
[112]Abe S, Inoue T. Fuzzy support vector machines for multiclass problems. In: Proceedings of the10th European Symposium on Artificial Neural Networks, Bruges, Belgium, April2002:113-118.
[113]Tsujinishi D, Abe S. Fuzzy least squares support vector machines for multiclass problems. Neural Networks,2003,16(5-6):785-792.
[114]Kijsirikul B, Ussivakul N. Multiclass support vector machines using adaptive directed acyclic graph. In: Proceedings of International Joint Conference on Neural Networks,2002,1:980-985.
[115]蔺志青,郭军.一种相似汉字的识别算法.中文信息学报,2002,16(5)：44-48.
[116]胡家中.手写印刷体汉字中近似字的区分.中文信息学报,1995,9(1)：37-41.
[117]王正群,叶晖,孙兴华,杨静宇.基于模糊方向特征的手写体汉字识别.模式识别与人工智能,2001,14(3)：317-320.
[118]张德喜,马少平,朱绍文,金弈江.基于统计与神经元方法相结合的手写体相似字识别.中文信息学报,1999,13(3)：33-39.
[119]梁曼君,石竹.基于神经网络的相似汉字识别的研究中文信息学报,1993,7(3)：26-32.
[120]田盛丰,黄厚宽,李洪波.基于支持向量机的手写体相似字识别.中文信息学报,2000,14(3)：37-41.
[121]Mizukami Y. A handwritten Chinese character recognition system using hierarchical displacement extraction based on directional features. Pattern Recognition Letters,1998,19(7):595-604.
[122]Sun N. A fine classification method of handwritten character by using automatic learning algorithm of partial area matching. Transaction of the Institute of Electronics, Information and Communication Engineers D-II,1995, J78-II (3):492-500.
[2]Vapnik V N. The nature of statistical learning theory. Springer-Verlag, London, UK,1995.
[3]Vapnik V N. Statistical learning theory. John Wiley&Sons, New York, USA,1998.
[4]Vapnik V N. The nature of statistical learning theory (Second Edition). Springer, New York, USA,1999.
[5]Tettamanzi A, Tomassim M. Soft computing: integrating evolutionary, neural, and fuzzy systems. Springer-Verlag, Berlin, Heidelberg,2001.
[6]Casey R, Nagy G. Recognition of printed Chinese character. IEEE Transaction on Electric Computer,1996,1(15):91-101.
[7]张忻中.我国汉字识别技术的历史、现状和展望.中文信息学报,1995,9(1)：25-30.
[8]赵宇明,江兴智,施鹏飞.基于笔划提取和合并的离线手写体汉字字符切分算法.红外与激光工程,2002,31(1)：23-27.
[9]周嫔,马少平,姜哲.基于单元合并的汉字切分算法的改进.中文信息学报,1999,13(2)：33-39.
[10]吕岳,施鹏飞,张克华.基于组件合并的手写体汉字串分割.软件学报,2000,11(11)：1154-1159.
[11]Zhao S Y, Chi Z R, Shi P F, Yan H. Two-stage segmentation unconstrained handwritten Chinese characters. Pattern Recognition,2003,36(1):145-156.
[12]朱小燕,王松.傅立叶变换在粘连文字图像切分中的应用.计算机学报,1999,22(12)：1246-1252.
[13]Tseng Y H, Lee H J. Recognition-based handwritten Chinese character segmentation using a probabilistic Viterbi algorithm. Pattern Recognition Letters,1999,20(8):791-806.
[14]赵斌,苏辉,夏绍玮.一种无约束手写体数字串分割方法.中文信息学报,1998,12(3)：21-28.
[15]Yamada H, Yamamoto K, Saito T. A nonlinear normalization method for handprinted Kanji character recognition: line density equalization. Pattern Recognition,1990,23(9):1023-1029.
[16]金连文,黄建成,尹俊勋,贺前华.变形变换及其在手写体汉字整形中的应用.中国图象图形学报,2002,7A(2)：170-175.
[17]王庆,赵荣椿,冯大淦.手写体汉字的规范化处理及评价.数据采集与处理,2001,16(2)：227-232.
[18]张睿,丁晓青,方驰.脱机手写汉字识别的最优采样特征新方法.中国图象图形学报,2002,7A(2)：176-180.
[19]马少平,夏莹,朱小燕.基于模糊方向线素特征的手写体汉字识别.清华大学学报(自然科学版),1997,37(3)：42-45.
[20]Lee H M, Huag C W, Sheu C C A fuzzy rule-based for handwritten Chinese characters recognition based on radical extraction. Fuzzy Sets and Systems,1998,100(1-3):59-70.
[21]陈津颖,金奕江,马少平.手写体汉字在特征空间的可视化分析.中文信息学报,2000,14(5)：42-48.
[22]张青,尹俊勋.小波变换在手写体汉字识别中的应用.电路与系统学报,1996,1(3)：63-67.
[23]Yu T, Ernest C M L, Tang Y Y. Feature extraction using wavelet and fractal. Pattern Recognition Letters,2001,22(3-4):271-287.
[24]金连文,彭秀兰,尹俊勋.一种手写体汉字特征提取新方法：小波变换及弹性网格技术的应用.中国图象图形学报,1998,3(7)：549-552.
[25]居琰,汪同庆,彭建.特征融合用于手写体汉字识别研究.电于科技大学学报,2002,31(3)：229-233.
[26]杨健,杨静宇,王正群,郭丽.一种组合特征抽取的新方法.计算机学报,2002,25(6)：570-575.
[27]王正群,孙兴华,郭丽,杨静宇.基于Fisher准则的多特征融合.计算机工程,2002,28(3)：41-42.
[28]金连文,梁宇杰.一种新的距离分类方法及其应用.计算机工程,1999,25(8)：30-31,57.
[29]盛立东,何其明.基于神经网络的手写体汉字识别实验系统.中文信息学报,1993,7(1)：48-57.
[30]邵秀丽,李勇建,蔡文进,刘憬.基干进化神经网络的手写体汉字识别.南开大学学报(自然科学版),2001,34(4)：53-56.
[31]吴鸣锐,张钹.一种用于大规模模式识别问题的神经网络算法.软件学报,2001,12(6)：851-855.
[32]金连文,徐秉铮RBF-DDA神经网络用于手写体汉字识别的研究.华南理工大学学报(自然科学版),1997,25(9)：97-101.
[33]Suganthan P N, Yan H. Recognition of handprinted Chinese characters by constrained graph matching. Image Vision Computing,1998,16(3):191-201.
[34]Chiu H P, Tseng D C. Invariant handwritten Chinese character recognition using fuzzy min-max neural networks. Pattern Recognition Letters,1997,18(5):481-491.
[35]金连文,徐秉铮.基于多级神经网络结构的手写体汉字识别.通信学报,1997,18(5)：21-27.
[36]韩宏,杨静宇.神经网络分类器的组合.计算机研究与发展,2000,37(12)：1488-1492.
[37]Devil H F Y, William W H Y. Different approach to designing neural network for similar handwritten Chinese character recognition. In: Proceedings of SPIE, Conference on Applications and Science of Neural Networks, Fuzzy Systems, and Evolutionary Computation,1998,3455:286-291.
[38]Liu C L, Kim I J, Kim J H. Model-based stroke extraction and matching for handwritten Chinese character recognition. Pattern Recognition,2001,34(12):2339-2352.
[39]Hsieh A J, Fan K C, Fan T I. Handwritten Chinese characters recognition by greedy matching with geometric constraint. Image Vision Computing,1996,14(2):91-104.
[40]高学,金连文,尹俊勋,黄建成.一种基于支持向量机的手写汉字识别方法.电子学报,2002,30(5)：651-654.
[41]范劲松,方廷健.基于粗集理论和SVM算法的模式分类方法.模式识别与人工智能,2000,13(4)：419-423.
[42]吕岳,施鹏飞,赵宇明.多分类器组合的投票表决规则.上海交通大学学报,2000,34(5)：680-683.
[43]孙怀江,胡钟山,杨静宇.基于证据理论的多分类器融合方法研究.计算机学报,2001,24(3)：231-235.
[44]征荆,丁晓青,吴佑寿.基于最小代价的多分类器动态集成.计算机学报,1999,22(2)：182-187.
[45]王春恒,肖柏华,戴汝为.基于串/并结合机制的集成型脱机手写汉字识别方法.模式识别与人工智能,1999,12(3)：255-261.
[46]韩宏,杨静宇,娄震.基于层次的分类器组合.南京理工大学学报,2002,26(1)：10-14.
[47]郭宏,丁晓青,郭繁夏,吴佑寿.汉字识别多分类器集成的新方法.清华大学学报(自然科学版),1997,37(10)：91-94.
[48]肖旭红,戴汝为.一种识别手写汉字的多分类器集成方法.自动化学报,1997,23(5)：621-627.
[49]蔡樱,盛立东.手写文稿识别的一种后处理方法和系统集成.中文信息学报,2000,14(3)：30-36.
[50]Chang C H. Simulated annealing clustering of Chinese words for contextual text recognition. Pattern Recognition Letters,1996,17(1):57-66.
[51]李元祥,丁晓青,刘长松.基于HMM的汉语文本识别后处理研究.中文信息学报,1999,13(4)：29-34.
[52]李元祥,丁晓青,吴佑寿.一种基于字词结合的汉字识别上下文处理新方法.计算机研究与发展,2002,39(7)：838-842.
[53]薛炳如,杨静宇,娄震,胡钟山.小类别数手写汉字建模.中文信息学报,1999,13(6)：34-40.
[54]石繁槐,童学锋.SVM在小字符集脱机手写体汉字识别中的应用研究计算机工程,2002,28(6)：187-189.
[55]朱小燕,史一凡.基于反馈的手写体字符识别方法的研究.计算机学报,2002,25(5)：476-482.
[56]姜珊,孙玉方.手写汉字的集群识别.中文信息学报,1998,12(4)：1-8.
[57]Cortes C, Vapnik V. Support vector networks. Machine Learning,1995,20(3):273-297.
[58]Osuna E, Freund R, Girosi F. An improved training algorithm for support vector machines. In: Proceedings of the1997IEEE Workshop on Neural Networks for Signal Processing VII,1997:276-285.
[59]Joachims T. Making large-scale SVM learning practical. In: Advances in Kernel Methods-Support Vector Learning, MIT Press, Cambridge, MA,1999:169-184.
[60]Platt J C. Fast training of support vetor machines using sequential minimal optimization. In: Advances in Kernel Methods-Support Vector Learning, MIT Press, Cambridge, MA,1999:185-208.
[61]Wahba G. Reproducing kernel Hilbert spaces-two brief reviews. Technical Report1079, Department of Statistics, University of Wisconsin-Madison, April2003:1-12.
[62]Platt J C, Cristianini N, Shawe-Taylor J. Large margin DAGs for multiclass classification. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA,2000,12:547-553.
[63]Dietterich T G, Bakiri G. Solving multi-class learning problems via error-correcting output codes. Journal of Artificial Intelligent Reasearch,1995,2:263-286.
[64]Hsu W, Lin C J. A comparision of methods for multiclass support vector machines. IEEE Transaction on Neural Networks,2002,13(2):415-425.
[65]Chapelle O, Vapnik V. Choosing multiple parameters for support vector machines. Machine Learning,2002,46(1-3):131-159.
[66]Cristianini N, Campbel C. Dynamically adapting kernels in support vector machines. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA,1999,11:204-210.
[67]Anguita D, Ridella S, Rivieccio F, Zunino R Hyperparameter design criteria for support vector classifers. Neurocomputing,2003,55:109-134.
[68]Duan K, Keerthi S S, Poo A N. Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing,2003,51:41-59.
[69]Gold C, Sollichb P. Model selection for support vector machine classifcation. Neurocomputing,2003,55:221-249.
[70]Chapelle O, Vapnik V N. Model selection for support vector machines. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA,2000,12:230-236.
[71]Ayat N E, Cheriet M, Remaki L, Suen C Y. KMOD-a new support vector machine kernel with moderate decreasing for pattern recognition. In: Proceedings of the6th International Conference on Document Analysis and Recognition,2001:1215-1219.
[72]张莉,周伟达,焦李成.一类新的支撑矢量机核.软件学报,2002,13(4)：713-718.
[73]Haasdonk B, Keysers D. Tangent distance kernels for support vector machines. In: Proceedings of the16th International Conference on Pattern Recognition,2002,2:864-868.
[74]张小云,刘允才.高斯核支撑向量机的性能分析.计算机工程,2003,29(8)：22-25.
[75]Wang W J, Xu Z B, Lu W Z, Zhang X Y. Determination of the spread parameter in the Gaussian kernel for classifcation and regression. Neurocomputing,2003,55:643-663.
[76]陈维桓,李兴校.黎曼几何引论.北京：北京大学出版社,2002.
[77]Amari S, Nagaoka H. Methods of information geometry. American Mathematical Society, USA,2000.
[78]Amari S, Wu S. Improving support vector machine classifiers by modifying kernel functions. Neural Networks,1999,12(6):783-789.
[79]Wu S, Amari S. Conformal transformation of kernel functions:a data-dependent way to improve support vector machine classifiers. Neural Processing Letters,2002,15(1):59-67.
[80]Luntz A, Brailovsky V. On estimation of characters obtained in statistical procedure of recognition. Technicheskaya Kibernetica,1969,3:1-20.
[81]Wahba G. Support vector machine, reproducing kernel Hilbert spaces and the randomize GACV. In: Advances in Kernel Methods-Support Vector Learning, MIT Press,1999:69-88.
[82]Wahba G, Lin Y, Zhang H. Generalize approximate cross validation for support vector machines. In: Advances in Large Margin Classifiers, MIT Press, Cambridge, MA,2000:297-311.
[83]Jaakola T S, Haussler D. Exploiting generative models in discriminative classifiers. In: Proceedings of the10th Conference on Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA,11,1999:487-493.
[84]邓乃扬,田英杰.数据挖掘中的新方法一支持向量机.北京：科学出版社,2004.
[85]周伟达,张莉,焦李成.一种改进的推广能力衡量准则.计算机学报,2003,26(5)：598-604.
[86]Joachims T. Estimating the generalizaition performance of an SVM efficientily. In: Proceedings of the17th International Conference on Machine Learning, San Franscisco, California,2000:431-438.
[87]Opper M, Winther O. Gaussian processes and svm: mean field and leave-one-out. In: Advances in Large Margin Classifiers, MIT Press, Cambridge, MA,2000:311-326.
[88]Momma M, Bennett K P. A pattern search method for model selection of support vector regression. In: Proceedings of the2th SIAM International Conference on Data Mining, Arlington, Virginia, April2002:261-274.
[89]Burges C J C. Geometry and invariance in kernel based methods. In: Advances in Kernel Methods-Support Vector Learning, Cambridge, MIT Press, Cambridge, MA,1999:89-116.
[90]周伟达,张莉,焦李成.支撑矢量机推广能力分析.电子学报,2001,29(5)：590-594.
[91]Barzilay O, Brailovsky V L. On domain knowledge and feature selection using a support vector machine. Pattern Recognition Letters,1999,20(5):475-484.
[92]Cao L J, Tay F E. Feature selection for support vector machines in financial time series forecasting. In: Proceedings of the2th International Conference on Intelligent Data Engineering and Automated Learning: Data Mining, Financial Engineering, and Intelligent Agents, Springer-Verlag, December, LNCS1983,2000:268-273.
[93]Kohavi R, John G H. Wrappers for feature subset selection. Artificial Intelligence,1997,(1-2):273-324.
[94]Zhang H, Sun G. Feature selection using tabu search method. Pattern Recognition,2002,35(3):701-711.
[95]Yamany S M, Khiani K J, Farag A A. Application of neural networks and genetic algorithms in the classification of endothelial cells. Pattern Recognition Letters,1997,18(11-13):1205-1210.
[96]Purshouse R C, Fleming P J. The multi-objective genetic algorithm applied to benchmark problems-an analysis. Research Report No.796, UK: University of Sheffield, Department of Automatic Control and Systems Engineering,2001,8:1-44.
[97]Horn J, Nafpliotis N, Goldberg D E. A niched pareto genetic algorithm for multiobjective optimization. In: Proceedings of the1th IEEE Conference on Evolutionary Computation, IEEE Press, Piscataway, NJ,1994:82-87.
[98]Goldberg D E. Genetic algorithms in search, optimization and machine learning. Reissue, Addison-Wesley Publishing Company,1989.
[99]Emmanouilidis C, Hunter A, Maclntyre J. A multiobjective evolutionary setting for feature selection and a commonality-based crossover operator. In: Proceedings of Congress on Evolutionary Computation,2000,1:309-316.
[100]Oliveira L S, Sabourin R, Bortolozzi F, Suen C Y. Feature selection using multi-objective genetic algorithms for handwritten digit recognition. In: Proceedings of16th International Conference on Pattern Recognition (ICPR),2002,1:568-571.
[101]Srinivas N, Deb K. Multiobjective optimization using nondominated sorting in genetic algorithms. Evolutionary Computation,1995,2(3):221-248.
[102]Lin C F, Wang S D. Fuzzy support vector machines. IEEE Transaction on Neural Networks,2002,13(3):464-471.
[103]Lin C F, Wang S D. Training algorithms for fuzzy support vector machines with noisy data. Pattern Recognition Letters,2004,25(14):1647-1656.
[104]Huang H P, Liu Y H. Fuzzy support vector machines for pattern recognition and data mining. International Journal of Fuzzy Systems,2002,4(3):826-835.
[105]Camps-Valls G, Martin-Guerrero J D, Rojo-Alvarez J L et al. Fuzzy sigmoid kernel for support vector classifiers. Neurocomputing,2004,62:501-506.
[106]Abe S. Analysis of multiclass support vector machines. In: Proceedings of International Conference on Computational Intelligence for Modeling Control and Automation, Vienna, Austria, Feburary2003:385-396.
[107]Inoue T, Abe S. Fuzzy support vector machines for pattern recognition. In: Proceedings of International Joint Conference on Neural Network, Washington, D.C., July2001,2:1449-1454.
[108]Tsang E C C, Yeung D S, Chan P P K. Fuzzy support vector machines for solving two-class problems. In: Proceedings of the2th International Conference on Machine Learning and Cybernetics, Xi'an, China, November2003,2:1080-1083.
[109]李昆仑,黄厚宽,田盛丰.模糊多类SVM模型.电子学报,2004,32(5)：830-832.
[110]Weston J, Watkins C. Support vector machines for multiclass pattern recognition. In: Proceedings of the7th European Symposium on Artificial Neural Networks, April1999:219-224.
[111]Kreβel U H-G. Pairwise classification and support vector machines. In: Advances in Kernel Methods:Support Vector Learning, MIT Press, Cambridge, MA,1999:255-268.
[112]Abe S, Inoue T. Fuzzy support vector machines for multiclass problems. In: Proceedings of the10th European Symposium on Artificial Neural Networks, Bruges, Belgium, April2002:113-118.
[113]Tsujinishi D, Abe S. Fuzzy least squares support vector machines for multiclass problems. Neural Networks,2003,16(5-6):785-792.
[114]Kijsirikul B, Ussivakul N. Multiclass support vector machines using adaptive directed acyclic graph. In: Proceedings of International Joint Conference on Neural Networks,2002,1:980-985.
[115]蔺志青,郭军.一种相似汉字的识别算法.中文信息学报,2002,16(5)：44-48.
[116]胡家中.手写印刷体汉字中近似字的区分.中文信息学报,1995,9(1)：37-41.
[117]王正群,叶晖,孙兴华,杨静宇.基于模糊方向特征的手写体汉字识别.模式识别与人工智能,2001,14(3)：317-320.
[118]张德喜,马少平,朱绍文,金弈江.基于统计与神经元方法相结合的手写体相似字识别.中文信息学报,1999,13(3)：33-39.
[119]梁曼君,石竹.基于神经网络的相似汉字识别的研究中文信息学报,1993,7(3)：26-32.
[120]田盛丰,黄厚宽,李洪波.基于支持向量机的手写体相似字识别.中文信息学报,2000,14(3)：37-41.
[121]Mizukami Y. A handwritten Chinese character recognition system using hierarchical displacement extraction based on directional features. Pattern Recognition Letters,1998,19(7):595-604.
[122]Sun N. A fine classification method of handwritten character by using automatic learning algorithm of partial area matching. Transaction of the Institute of Electronics, Information and Communication Engineers D-II,1995, J78-II (3):492-500.