基于多特征卷积神经网络的手写公式符号识别
|
摘要
提出了基于多特征稠密卷积神经网络的模型框架(DenseNet-SE)。与传统方法相比,DenseNet-SE采用数据驱动的方法,无需手工提取特征。该框架包含了稠密残差块的结构,能够获取深度特征。通过跳跃连接的方式,从浅层获取细粒度特征来辅助深度特征。同时,融合特征有助于网络结构获取更多全局信息,更好地表示公式符号的类别。利用在线手写数学表达式识别的竞赛组织(CROHME)提供的标准数学公式符号库来验证所提算法,结果表明,CROHME2014和CROHME2016的识别率分别达到93.38%和92.93%,高于目前已有算法的识别率。
A model framework called DenseNet-SE is proposed based on a multi-featured dense convolutional neural network.Compared with the conventional methods,the DenseNet-SE adopts the data-driven approach and the manual extraction of features is not necessary.It contains the dense residual blocks so that the deep features can be acquired.In the jump-joining way,the fine-grained features are obtained from the shallow layers to assist the deep features.The fused features can help the network structure obtain more global information and better represent the categories of formula symbols.The standard mathematical formula symbol library provided by the competition organization on recognition of online handwritten mathematical expression(CROHME)is used to verify the proposed algorithm,results show that the recognition rates of CROHME2014 and CROHME2016 are 93.38% and92.93%,respectively,higher than those of the existing algorithms.
A model framework called DenseNet-SE is proposed based on a multi-featured dense convolutional neural network.Compared with the conventional methods,the DenseNet-SE adopts the data-driven approach and the manual extraction of features is not necessary.It contains the dense residual blocks so that the deep features can be acquired.In the jump-joining way,the fine-grained features are obtained from the shallow layers to assist the deep features.The fused features can help the network structure obtain more global information and better represent the categories of formula symbols.The standard mathematical formula symbol library provided by the competition organization on recognition of online handwritten mathematical expression(CROHME)is used to verify the proposed algorithm,results show that the recognition rates of CROHME2014 and CROHME2016 are 93.38% and92.93%,respectively,higher than those of the existing algorithms.
引文
[1]Blostein D,Grbavec A.Recognition of mathematical notation[M].Singapore:World Scientific,1997:557-582.
[2]Chan K F,Yeung D Y.Mathematical expression recognition:a survey[J].International Journal on Document Analysis and Reconition,2000,3(1):3-15.
[3]lvaro F,Sánchez J A,BenedíJ M.Offline features for classifying handwritten math symbols with recurrent neural networks[C]∥International Conference on Pattern Recognition,2014:2944-2949.
[4]Davila K,Ludi S,Zanibbi R.Usingoff-line features and synthetic data for on-line handwritten math symbol recognition[C]∥International Conference on Frontiers in Handwriting Recognition,2014:323-328.
[5]Nguyen H D,Duc le A,Nakagawa M.Recognition of online handwritten math symbols using deep neural networks[J].IEICE Transactions on Information and Systems,2016,E99.D(12):3110-3118.
[6]Mouchère H,Viard-Gaudin C,Zanibbi R,et al.ICFHR2016 CROHME:competition on recognition of online handwritten mathematical expressions[C]∥International Conference on Frontiers in Handwriting Recognition,2016:607-612.
[7]Dong L F,Liu H C.Recognition of offline handwritten mathematical symbols using convolutional neural networks[M].Cham:Springer International Publishing,2017:149-161.
[8]Ramadhan I,Purnama B,Faraby S A.Convolutional neural networks applied to handwritten mathematical symbols classification[C]∥International Conference on Information and Communication Technology,2016:1-4.
[9]Feng X Y,Mei W,Hu D S.Aerial target detection based on improved faster R-CNN[J].Acta Optica Sinica,2018,38(6):0615004.冯小雨,梅卫,胡大帅.基于改进Faster R-CNN的空中目标检测[J].光学学报,2018,38(6):0615004.
[10]Guo C C,Yu F Q,Chen Y.Image semantic segmentation based on convolutional neural network feature and improved superpixel matching[J].Laser&Optoelectronics Progress,2018,55(8):081005.郭呈呈,于凤芹,陈莹.基于卷积神经网络特征和改进超像素匹配的图像语义分割[J].激光与光电子学进展,2018,55(8):081005.
[11]Wang M,Liu K X,Liu L,et al.Super-resolution reconstruction of image based on optimized convolution neural network[J].Laser&Optoelectronics Progress,2017,54(11):111005.王民,刘可心,刘利,等.基于优化卷积神经网络的图像超分辨率重建[J].激光与光电子学进展,2017,54(11):111005.
[12]LeCun Y,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[13]Hinton G E,Srivastava N,Krizhevsky A,et al.Improving neural networks by preventing coadaptation of feature detectors[J].arXiv preprint arXiv:1207.0580,2012.
[14]He K M,Zhang X Y,Ren S Q,et al.Deep residual learning for image recognition[C]∥IEEE Conference on Computer Vision and Pattern Recognition,2016:770-778.
[15]Huang G,Liu Z,Maaten L V D,et al.Densely connected convolutional networks[C]∥IEEEConference on Computer Vision and Pattern Recognition,2017:2261-2269.
[16]W3C Recommendation.Ink makeup language[EB/OL].(2017-04-06)[2018-08-06].http:∥www.w3.org/TR/InkML/.
[17]Simard P Y,Steinkraus D,Platt J C.Bestpractices for convolutional neural networks applied to visual document analysis[C]∥International Conference on Document Analysis and Recognition,2003:958-963.
[18]Hu J,Shen L,Sun G.Squeeze-and-excitation networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:7132-7141.
[19]Ioffe S,Szegedy C.Batchnormalization:accelerating deep network training by reducing internal covariate shift[C]∥32nd International Conference on Machine Learning,2015,37:448-456.
[20]Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[21]Mouchère H,Zanibbi R,Garain U,et al.Advancing the state of the art for handwritten math recognition:the CROHME competitions,2011-2014[J].International Journal on Document Analysis and Recognition,2016,19(2):173-189.
[22]Mouchère H,Viard-Gaudin C,Zanibbi R,et al.ICFHR 2014competition on recognition of on-line handwritten mathematical expressions(CROHME2014)[C]∥International Conference on Frontiers in Handwriting Recognition,2014:791-796.
[23]Kingma D P,Ba J.Adam:a method for stochastic optimization[J].arXiv preprint arXiv:1412.6980,2014.
[2]Chan K F,Yeung D Y.Mathematical expression recognition:a survey[J].International Journal on Document Analysis and Reconition,2000,3(1):3-15.
[3]lvaro F,Sánchez J A,BenedíJ M.Offline features for classifying handwritten math symbols with recurrent neural networks[C]∥International Conference on Pattern Recognition,2014:2944-2949.
[4]Davila K,Ludi S,Zanibbi R.Usingoff-line features and synthetic data for on-line handwritten math symbol recognition[C]∥International Conference on Frontiers in Handwriting Recognition,2014:323-328.
[5]Nguyen H D,Duc le A,Nakagawa M.Recognition of online handwritten math symbols using deep neural networks[J].IEICE Transactions on Information and Systems,2016,E99.D(12):3110-3118.
[6]Mouchère H,Viard-Gaudin C,Zanibbi R,et al.ICFHR2016 CROHME:competition on recognition of online handwritten mathematical expressions[C]∥International Conference on Frontiers in Handwriting Recognition,2016:607-612.
[7]Dong L F,Liu H C.Recognition of offline handwritten mathematical symbols using convolutional neural networks[M].Cham:Springer International Publishing,2017:149-161.
[8]Ramadhan I,Purnama B,Faraby S A.Convolutional neural networks applied to handwritten mathematical symbols classification[C]∥International Conference on Information and Communication Technology,2016:1-4.
[9]Feng X Y,Mei W,Hu D S.Aerial target detection based on improved faster R-CNN[J].Acta Optica Sinica,2018,38(6):0615004.冯小雨,梅卫,胡大帅.基于改进Faster R-CNN的空中目标检测[J].光学学报,2018,38(6):0615004.
[10]Guo C C,Yu F Q,Chen Y.Image semantic segmentation based on convolutional neural network feature and improved superpixel matching[J].Laser&Optoelectronics Progress,2018,55(8):081005.郭呈呈,于凤芹,陈莹.基于卷积神经网络特征和改进超像素匹配的图像语义分割[J].激光与光电子学进展,2018,55(8):081005.
[11]Wang M,Liu K X,Liu L,et al.Super-resolution reconstruction of image based on optimized convolution neural network[J].Laser&Optoelectronics Progress,2017,54(11):111005.王民,刘可心,刘利,等.基于优化卷积神经网络的图像超分辨率重建[J].激光与光电子学进展,2017,54(11):111005.
[12]LeCun Y,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[13]Hinton G E,Srivastava N,Krizhevsky A,et al.Improving neural networks by preventing coadaptation of feature detectors[J].arXiv preprint arXiv:1207.0580,2012.
[14]He K M,Zhang X Y,Ren S Q,et al.Deep residual learning for image recognition[C]∥IEEE Conference on Computer Vision and Pattern Recognition,2016:770-778.
[15]Huang G,Liu Z,Maaten L V D,et al.Densely connected convolutional networks[C]∥IEEEConference on Computer Vision and Pattern Recognition,2017:2261-2269.
[16]W3C Recommendation.Ink makeup language[EB/OL].(2017-04-06)[2018-08-06].http:∥www.w3.org/TR/InkML/.
[17]Simard P Y,Steinkraus D,Platt J C.Bestpractices for convolutional neural networks applied to visual document analysis[C]∥International Conference on Document Analysis and Recognition,2003:958-963.
[18]Hu J,Shen L,Sun G.Squeeze-and-excitation networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:7132-7141.
[19]Ioffe S,Szegedy C.Batchnormalization:accelerating deep network training by reducing internal covariate shift[C]∥32nd International Conference on Machine Learning,2015,37:448-456.
[20]Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[21]Mouchère H,Zanibbi R,Garain U,et al.Advancing the state of the art for handwritten math recognition:the CROHME competitions,2011-2014[J].International Journal on Document Analysis and Recognition,2016,19(2):173-189.
[22]Mouchère H,Viard-Gaudin C,Zanibbi R,et al.ICFHR 2014competition on recognition of on-line handwritten mathematical expressions(CROHME2014)[C]∥International Conference on Frontiers in Handwriting Recognition,2014:791-796.
[23]Kingma D P,Ba J.Adam:a method for stochastic optimization[J].arXiv preprint arXiv:1412.6980,2014.