基于多任务卷积神经网络的舌象分类研究
|
摘要
针对现有技术难以并行实现舌象多标签的高效分类和识别,难以利用标签间的相关性进行综合分析等问题,提出了一种基于多任务卷积神经网络的舌象分类方法,构建了一种多任务联合学习模型,尝试实现传统中医舌诊中对舌色、苔色、裂纹和齿痕等多个标签的同时辨识。首先,在共享网络层对所有标签进行联合学习,从特征提取的角度自动挖掘和利用标签间的相关性;然后,在不同子网络层分别完成特定类别的学习任务,从而消除多标签分类中的歧义性;最后,训练多个Softmax分类器以实现对所有标签的并行预测。研究表明,所提方法能以端到端的方式同时提取舌象的多个特征并直接进行分类识别,在各分类评价指标上的最低值约为0.96,多任务的总体识别时间为34ms,因此该方法在精度和速度上均具有明显优势。
It is difficult to exploit the existing methods to achieve efficient classification and identification of tongue image'labels in parallel,and it is also difficult to utilize the correlation between labels for comprehensive analysis.Aiming at the problems above,this paper proposed a classification method of tongue image based on multi-task deep convolutional neural network and constructed a multi-task joint learning model based on deep convolutional neural network to realize the simultaneous identification of tongue color,moss color,tongue crack and tooth marks in tongue diagnosis of Chinese medicine.First,the shared network layer is used to learn all labels,and the correlation between the tags is extracted and utilized automatically from the perspective of feature extraction.Then,the learning tasks of specific labels are completed in different sub-network layers to eliminate the ambiguity in the multi-label classification problem.Finally,multiple Softmax classifiers are trained to achieve parallel prediction of all labels.Experimental results suggest that the proposed method can simultaneous extract multiple features of tongue image and implement classification by means of end to end.The lowest value is about 0.96 in several evaluation indexes and the multi-task recognition rate is about34 ms.Therefore,this algorithm has obvious advantages in accuracy and speed.
It is difficult to exploit the existing methods to achieve efficient classification and identification of tongue image'labels in parallel,and it is also difficult to utilize the correlation between labels for comprehensive analysis.Aiming at the problems above,this paper proposed a classification method of tongue image based on multi-task deep convolutional neural network and constructed a multi-task joint learning model based on deep convolutional neural network to realize the simultaneous identification of tongue color,moss color,tongue crack and tooth marks in tongue diagnosis of Chinese medicine.First,the shared network layer is used to learn all labels,and the correlation between the tags is extracted and utilized automatically from the perspective of feature extraction.Then,the learning tasks of specific labels are completed in different sub-network layers to eliminate the ambiguity in the multi-label classification problem.Finally,multiple Softmax classifiers are trained to achieve parallel prediction of all labels.Experimental results suggest that the proposed method can simultaneous extract multiple features of tongue image and implement classification by means of end to end.The lowest value is about 0.96 in several evaluation indexes and the multi-task recognition rate is about34 ms.Therefore,this algorithm has obvious advantages in accuracy and speed.
引文
[1] LEE J,KIM H,KIM N R,et al.An approach for multi-label classification by directed acyclic graph with label correlation maximization[J].Information Sciences,2016,351(C):101-114.
[2] WU J,YE C,SHENG V S,et al.Active learning with label correlation exploration for multi-label image classification[J].Iet Computer Vision,2017,11(7):577-584.
[3] LI S L,LIU R,LIU H.Multi-Label Learning for Improved RBF Neural Networks[J].Computer Science,2015,42(4):316-320.(in Chinese)李书玲,刘蓉,刘红.改进型RBF神经网络的多标签算法研究[J].计算机科学,2015,42(4):316-320.
[4] XU X D,YAO M H,LIU H W,et al.Pre-processing Method of Multi-label Classification Based on kNN[J].Computer Science,2015,42(5):106-108,131.(in Chinese)徐晓丹,姚明海,刘华文,等.基于kNN的多标签分类预处理方法[J].计算机科学,2015,42(5):106-108,131.
[5] ZHANG Y F,HU G Q,ZHANG X F,et al.An algorithm study on tongue color recognition of patients[J].Beijing Biomedical Engineering,2016,35(1):7-11.(in Chinese)张艺凡,胡广芹,张新峰,等.基于支持向量机的痤疮患者舌色苔色识别算法研究[J].北京生物医学工程,2016,35(1):7-11.
[6] WANG Y G,YANG J,ZHOU Y,et al.Tongue Image Color Recognition in Traditional Chinese Medicine[J].Journal of Biomedical Engineering,2005,22(6):1116-1120.(in Chinese)王永刚,杨杰,周越,等.中医舌象颜色识别的研究[J].生物医学工程学杂志,2005,22(6):1116-1120.
[7] YANG Z H,ZHANG D P,LI N M.Kernel False-Colour Transformation and Line Extraction for Fissured Tongue Image[J].Journal of Computer-Aided Design and Computer Graphics,2010,22(5):771-776.(in Chinese)杨朝辉,张大鹏,李乃民.裂纹舌图像的核假彩色变换及其纹线提取[J].计算机辅助设计与图形学学报,2010,22(5):771-776.
[8] ZHUMU L M,LU P,XIA C M,et al.Research on Douglas-Peucker Method in Feature Extraction from 55 Cases of ToothMarked Tongue Image[J].Chinese Archives of Traditional Chinese Medicine,2014,32(9):2138-2140.(in Chinese)朱穆朗玛,陆萍,夏春明,等.基于道格拉斯-普克法提取55例齿痕舌图像特征研究[J].中华中医药学刊,2014,32(9):2138-2140.
[9] LIU W,ANGUELOV D,ERHAN D,et al.SSD:Single shot multibox detector[C]∥Proceedings of European Conference on Computer Vision.Amsterdam:Springer International Publishing,2016:21-37.
[10]JONATHAN L,EVAN S,TREVOR D.Fully convolutional networks for semantic segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE,2015:3431-3440.
[11]SUN Y,WANG X G,TANG X O.Deep Learning Face Representation from Predicting 10,000Classes[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus:IEEE,2014:1891-1898.
[12]GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus:IEEE,2014:580-587.
[13]GIRSHICK R.Fast R-CNN[C]∥Proceedings of the IEEE Conference on International Conference on Computer Vision.Boston:IEEE,2015:1440-1448.
[14]REN S Q,HE K M,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2017,39(6):1137-1149.
[15]HOOCHANG S,ROTH H R,GAO M,et al.Deep convolutional neural networks for computer-aided detection:CNN architectures,dataset characteristics and transfer learning[J].IEEE Transactions on Medical Imaging,2016,35(5):1285-1298.
[16]DONAHUE J,JIA Y Q,VINYALS O,et al.DeCAF:A Deep Convolutional Activation Feature for Generic Visual Recognition[C]∥Proceedings of the 31st International Conference on Machine Learning.2014:I-647-I-655.
[17]RAZAVIAN A S,AZIZPOUR H,SULLIVAN J,et al.CNN features off-the-shelf:An Astounding Baseline for Recognition[C]∥2014IEEE Conference on Computer Vision and Pattern Recognition Workshops(CV PRW).2014:512-519.
[18]YOSINSKI J,CLUNE J,BENGIO Y,et al.How transferable are features in deep neural networks?[C]∥Proceedings of the IEEE Conference on Neural Information Processing Systems.Montreal:MIT Press,2014:3320-3328.
[19]DENG J,DONG W,SOCHER R,et al.ImageNet:A large-scale hierarchical image database[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Miami:IEEE,2009:248-255.
[20]KAN H X,ZHANG L Y,DONG C W.A Tongue Image Recognition Method Based on Type 2 Diabetes Traditional Chinese Medicine Syndrome Classification[J].Chinese Journal of Biomedical Engineering,2016,35(6):658-664.(in Chinese)阚红星,张璐瑶,董昌武.一种2型糖尿病中医证型的舌图像识别方法[J].中国生物医学工程学报,2016,35(6):658-664.
[2] WU J,YE C,SHENG V S,et al.Active learning with label correlation exploration for multi-label image classification[J].Iet Computer Vision,2017,11(7):577-584.
[3] LI S L,LIU R,LIU H.Multi-Label Learning for Improved RBF Neural Networks[J].Computer Science,2015,42(4):316-320.(in Chinese)李书玲,刘蓉,刘红.改进型RBF神经网络的多标签算法研究[J].计算机科学,2015,42(4):316-320.
[4] XU X D,YAO M H,LIU H W,et al.Pre-processing Method of Multi-label Classification Based on kNN[J].Computer Science,2015,42(5):106-108,131.(in Chinese)徐晓丹,姚明海,刘华文,等.基于kNN的多标签分类预处理方法[J].计算机科学,2015,42(5):106-108,131.
[5] ZHANG Y F,HU G Q,ZHANG X F,et al.An algorithm study on tongue color recognition of patients[J].Beijing Biomedical Engineering,2016,35(1):7-11.(in Chinese)张艺凡,胡广芹,张新峰,等.基于支持向量机的痤疮患者舌色苔色识别算法研究[J].北京生物医学工程,2016,35(1):7-11.
[6] WANG Y G,YANG J,ZHOU Y,et al.Tongue Image Color Recognition in Traditional Chinese Medicine[J].Journal of Biomedical Engineering,2005,22(6):1116-1120.(in Chinese)王永刚,杨杰,周越,等.中医舌象颜色识别的研究[J].生物医学工程学杂志,2005,22(6):1116-1120.
[7] YANG Z H,ZHANG D P,LI N M.Kernel False-Colour Transformation and Line Extraction for Fissured Tongue Image[J].Journal of Computer-Aided Design and Computer Graphics,2010,22(5):771-776.(in Chinese)杨朝辉,张大鹏,李乃民.裂纹舌图像的核假彩色变换及其纹线提取[J].计算机辅助设计与图形学学报,2010,22(5):771-776.
[8] ZHUMU L M,LU P,XIA C M,et al.Research on Douglas-Peucker Method in Feature Extraction from 55 Cases of ToothMarked Tongue Image[J].Chinese Archives of Traditional Chinese Medicine,2014,32(9):2138-2140.(in Chinese)朱穆朗玛,陆萍,夏春明,等.基于道格拉斯-普克法提取55例齿痕舌图像特征研究[J].中华中医药学刊,2014,32(9):2138-2140.
[9] LIU W,ANGUELOV D,ERHAN D,et al.SSD:Single shot multibox detector[C]∥Proceedings of European Conference on Computer Vision.Amsterdam:Springer International Publishing,2016:21-37.
[10]JONATHAN L,EVAN S,TREVOR D.Fully convolutional networks for semantic segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE,2015:3431-3440.
[11]SUN Y,WANG X G,TANG X O.Deep Learning Face Representation from Predicting 10,000Classes[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus:IEEE,2014:1891-1898.
[12]GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus:IEEE,2014:580-587.
[13]GIRSHICK R.Fast R-CNN[C]∥Proceedings of the IEEE Conference on International Conference on Computer Vision.Boston:IEEE,2015:1440-1448.
[14]REN S Q,HE K M,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2017,39(6):1137-1149.
[15]HOOCHANG S,ROTH H R,GAO M,et al.Deep convolutional neural networks for computer-aided detection:CNN architectures,dataset characteristics and transfer learning[J].IEEE Transactions on Medical Imaging,2016,35(5):1285-1298.
[16]DONAHUE J,JIA Y Q,VINYALS O,et al.DeCAF:A Deep Convolutional Activation Feature for Generic Visual Recognition[C]∥Proceedings of the 31st International Conference on Machine Learning.2014:I-647-I-655.
[17]RAZAVIAN A S,AZIZPOUR H,SULLIVAN J,et al.CNN features off-the-shelf:An Astounding Baseline for Recognition[C]∥2014IEEE Conference on Computer Vision and Pattern Recognition Workshops(CV PRW).2014:512-519.
[18]YOSINSKI J,CLUNE J,BENGIO Y,et al.How transferable are features in deep neural networks?[C]∥Proceedings of the IEEE Conference on Neural Information Processing Systems.Montreal:MIT Press,2014:3320-3328.
[19]DENG J,DONG W,SOCHER R,et al.ImageNet:A large-scale hierarchical image database[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Miami:IEEE,2009:248-255.
[20]KAN H X,ZHANG L Y,DONG C W.A Tongue Image Recognition Method Based on Type 2 Diabetes Traditional Chinese Medicine Syndrome Classification[J].Chinese Journal of Biomedical Engineering,2016,35(6):658-664.(in Chinese)阚红星,张璐瑶,董昌武.一种2型糖尿病中医证型的舌图像识别方法[J].中国生物医学工程学报,2016,35(6):658-664.