Convolutional feature learning and Hybrid CNN-HMM for scene number recognition

详细信息    查看全文

• 作者：Qiang Guo ; ^{guoqiang05@nudt.edu.cn" class="auth_mail" title="E-mail the corresponding author}Author Vitae ; Fenglei Wang ^{wangfenglei@nudt.edu.cn" class="auth_mail" title="E-mail the corresponding author}Author Vitae ; Jun Lei ^{leijun1987@gmail.com" class="auth_mail" title="E-mail the corresponding author}Author Vitae ; Dan Tu ^{tudan@nudt.edu.cn" class="auth_mail" title="E-mail the corresponding author}Author Vitae ; Guohui Li ^{guohli@nudt.edu.cn" class="auth_mail" title="E-mail the corresponding author}Author Vitae
• 关键词：Scene text recognition ; Convolutional neural network ; Hidden Markov model ; Deep learning ; Hybrid NN-HMM ; GMM-HMM
• 刊名：Neurocomputing
• 出版年：2016
• 出版时间：5 April 2016
• 年：2016
• 卷：184
• 期：Complete
• 页码：78-90
• 全文大小：2669 K

文摘

In this work, we investigate to recognize house numbers captured in street view images. We formulate the problem as sequence recognition and present an integrated model by combining Convolutional Neural Network (CNN) and Hidden Markov Model (HMM). Our method utilizes representation capability of CNN to model the highly variable appearance of digits. Meanwhile, HMM is used to handle the dynamics of the image sequence. They are combined in a hybrid way to form the Hybrid CNN-HMM. Using this model, we can perform training and recognition both at the whole image level without explicit segmentation. The model makes CNN applicable to dynamic problems. Experiments show that the Hybrid CNN-HMM can dramatically boost the performance of Gaussian Mixture Model (GMM)-HMM. We evaluate different local features, e.g. LBP, SIFT and HOG, as observations fed into HMM and find CNN features consistently surpass those hand-engineered features with respect to recognition accuracy. To gain insight into performance difference of the features, we map them from the high-dimensional space to a 2-D plane by the t-SNE algorithm to visualize their semantic clustering with respect to the task. The visualization clearly justified the efficiency of features learnt by CNN.