Improving discrimination ability of convolutional neural networks by hybrid learning

详细信息    查看全文

• 作者：In-Jung Kim ; Changbeom Choi ; Sang-Heon Lee
• 关键词：Deep learning ; Convolutional neural networks ; Hybrid learning ; Discrimination ; Character recognition ; Machine learning ; Pattern recognition
• 刊名：International Journal on Document Analysis and Recognition
• 出版年：2016
• 出版时间：March 2016
• 年：2016
• 卷：19
• 期：1
• 页码：1-9
• 全文大小：792 KB
• 参考文献：1.Kim, I.-J., Kim, J.: Pairwise discrimination based on a stroke importance measure. Pattern Recognit. 35(10), 2259–2266 (2002)CrossRef MATH
  2.Leung, K.C., Leung, C.H.: Recognition of handwritten Chinese characters by critical region analysis. Pattern Recognit. 43(3), 949–961 (2010)CrossRef MATH
  3.Xu, B., Huang, K., Liu, C.L.: Similar characters recognition by critical region selection based on average symmetric uncertainty. In: Proceedings of 12th ICFHR, Kolkata, India, pp. 527–532 (2010)
  4.Ryu, S.-J., Kim, I.-J.: Discrimination of similar characters using nonlinear normalization based on regional importance measure. Int. J. Doc. Anal. Recognit. (IJDAR) 17(1), 79–89 (2014)CrossRef
  5.LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRef
  6.Kim, I.-J., Xie, X.: Handwritten Hangul recognition using deep convolutional neural networks. Int. J. Doc. Anal. Recognit. 18(1), 1–13 (2014)CrossRef
  7.Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition, arXiv:​1409.​1556 (2014)
  8.Yin, F., Wang, Q.F., Zhang, X.X.Y., Liu, C.L.: ICDAR 2013 Chinese handwriting recognition competition. http://​www.​nlpr.​ia.​ac.​cn/​events/​CHRcompetition20​13/​competition/​ICDAR%20​2013%20​CHR%20​competition.​pdf
  9.Kim, D.-I., Kim, S.-Y., Lee, S.-W.: Design and construction of a large-set off-line handwritten hangul character image database KU-1. In: Proceedings of \(9{{\rm th}}\) National Conference on Korean Language Information Processing, pp. 152–159 (1997) (in Korean)
  10.Kim, D.H., Bang, S.Y.: An overview of hangul handwritten image database PE92. In: Proceedings of 4\({{\rm th}}\) National Conference on Korean Language Information Processing, pp. 152–159 (1992) (in Korean)
  11.Kim, I.-J., Kim, J.: Pairwise discrimination based on a stroke importance measure. Pattern Recognit. 35(10), 2259–2266 (2002)
  12.Goodfellow, I. et al.: Maxout networks, arXiv:​1302.​4389 (2013)
  13.Lin, M., Chen, Q., Yan, S.: Network in network. In: Proceedings of ICLR (2014)
  14.Szegedy, C. et al.: Going deeper with convolutions, arXiv:​1409.​4842 (2014)
  15.Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift, arXiv:​1502.​03167 (2015)
  16.Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network, arXiv:​1503.​02531 (2015)
  17.Girshick, R. et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE (2014)
  18.Socher, R. et al.: Convolutional-recursive deep learning for 3d object classification. Advances in Neural Information Processing Systems (2012)
  19.Zeiler, M., Fergus, R.: Visualizing and understanding convolutional networks. Compu Vis-ECCV 2014, 818–833 (2014)
  20.Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps, arXiv:​1312.​6034 (2013)
  21.Sato, A., Yamada, K.: Generalized learning vector quantization. Adv. Neural Inf. Process. Syst. 15(8), 423–429 (1996)
  22.Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997)CrossRef MathSciNet
  23.Bengio, Y. et al.: Curriculum learning. In: Proceedings of 26th Annual International Conference on Machine Learning. ACM, pp. 41–48 (2009)
  24.Simard, P., Steinkraus, D., Platt, J.: Best practices for convolutional neural networks applied to visual document analysis, null. IEEE, p. 958 (2003)
  25.Srivastava, N., et al.: Dropout: a simple way to prevent neural networks from overfitting. J Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNet MATH
  
• 作者单位：In-Jung Kim (1)
  Changbeom Choi (2)
  Sang-Heon Lee (3)
  
  1. School of CSEE, Handong Global University, Pohang, Korea
  2. School of Creative Convergence Education, Handong Global University, Pohang, Korea
  3. Department of IoT and Robotics Convergence Research, DGIST, Daegu, Korea
  
• 刊物类别：Computer Science
• 刊物主题：Image Processing and Computer Vision
  Pattern Recognition
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1433-2825

文摘

The discrimination of similar patterns is important because they are the major sources of the classification error. This paper proposes a novel method to improve the discrimination ability of convolutional neural networks (CNNs) by hybrid learning. The proposed method embeds a collection of discriminators as well as a recognizer in a shared CNN. By visualizing contrastive class saliency, we show that learning with embedded discriminators leads the shared CNN to detect and catch the differences among similar classes. Also proposed is a hybrid learning algorithm that learns recognition and discrimination together. The proposed method learns recognition focusing on the differences among similar classes, and thereby improves the discrimination ability of the CNN. Unlike conventional discrimination methods, the proposed method does not require predefined sets of similar classes or additional step to integrate its result with that of the recognizer. In experiments on two handwritten Hangul databases SERI95a and PE92, the proposed method reduced classification error from 2.56 to 2.33, and from 4.04 to 3.66 % respectively. These improvement lead to relative error reduction rates of 8.97 % on SERI95a, and 9.42 % on PE92. Our best results update the state-of-the-art performance which were 4.04 % on SERI95a and 7.08 % on PE92.