基于层级循环神经网络的术中X线图像腰椎自动识别
|
摘要
针对图像引导微创脊柱手术中移动C型臂X线成像特点,通过学习人体腰椎的曲率特征实现腰椎识别,提出一种基于层级循环神经网络的X线图像腰椎自动识别方法.首先为解决X线图像中腰椎纹理混叠的问题,提取腰椎三维模型与二维X线图像中共有的曲率特征作为模型的输入;其次为模拟术中移动C型臂多角度成像的特点,采用双向循环神经网络学习腰椎曲率特征,刻画腰椎曲率特征在不同成像角度下的关联性;最后为解决病理情况下腰椎部分信息缺失的问题,提出一种层级循环神经网络模型,通过逐层融合的网络架构对人体腰椎间天然的上下文关系进行建模,提高模型在病理情况下的腰椎识别率.在开源数据集和术中移动C型臂X线图像上的实验结果表明,文中方法在正常情况和病理情况下的腰椎识别率均优于其他4种方法,且由于使用了数据量较少的二维曲率特征,该方法在训练和测试阶段的计算效率更高,更适合于术中图像引导的应用.
According to the characteristic of mobile C-arm X-ray imaging in image-guided minimally invasive spine surgery, an automatic lumbar vertebrae recognition method is proposed, which based on hierarchical recurrent neural network. Its purpose is to identify lumbar vertebrae automatically by learning the curvature features. First, in order to solve the problem of lumbar vertebrae texture overlapping in X-ray images, the curvature features of 3D lumbar vertebrae model, which are common to the 2D X-ray images, are taken as the input of the model. Second, in order to simulate the multi-view imaging of intraoperative C-arm, the bidirectional recurrent neural network is exploited to learn the correlation of lumbar curvature features at different imaging angles. Finally, in order to solve the problem of partial occlusion of the lumbar vertebrae in the pathological condition, a hierarchical recurrent neural network model is introduced. The natural context between human lumbar vertebrae is modeled by the layer-by-layer fusion architecture to improve the recognition rate in the pathological condition. The results of the verification on open source datasets and intraoperative mobile C-arm X-ray images show that the lumbar vertebrae recognition rate of the proposed method is superior to the other four methods in both normal and pathological conditions. Furthermore, due to the utilization of two-dimensional curvature features, the proposed method is more efficient in the training and testing phases, and more suitable for applications in intraoperative image-guided navigation.
According to the characteristic of mobile C-arm X-ray imaging in image-guided minimally invasive spine surgery, an automatic lumbar vertebrae recognition method is proposed, which based on hierarchical recurrent neural network. Its purpose is to identify lumbar vertebrae automatically by learning the curvature features. First, in order to solve the problem of lumbar vertebrae texture overlapping in X-ray images, the curvature features of 3D lumbar vertebrae model, which are common to the 2D X-ray images, are taken as the input of the model. Second, in order to simulate the multi-view imaging of intraoperative C-arm, the bidirectional recurrent neural network is exploited to learn the correlation of lumbar curvature features at different imaging angles. Finally, in order to solve the problem of partial occlusion of the lumbar vertebrae in the pathological condition, a hierarchical recurrent neural network model is introduced. The natural context between human lumbar vertebrae is modeled by the layer-by-layer fusion architecture to improve the recognition rate in the pathological condition. The results of the verification on open source datasets and intraoperative mobile C-arm X-ray images show that the lumbar vertebrae recognition rate of the proposed method is superior to the other four methods in both normal and pathological conditions. Furthermore, due to the utilization of two-dimensional curvature features, the proposed method is more efficient in the training and testing phases, and more suitable for applications in intraoperative image-guided navigation.
引文
[1]Baka N,Leenstra S,van Walsum T.Ultrasound aided vertebral level localization for lumbar surgery[J].IEEE Transactions on Medical Imaging,2017,36(10):2138-2147
[2]Zhang Peng,Xu Xinnan,Wang Hongwei,et al.Computer-aided lung cancer diagnosis approaches based on deep learning[J].Journal of Computer-Aided Design&Computer Graphics,2018,30(1):90-99(in Chinese)(张鹏,徐欣楠,王洪伟,等.基于深度学习的计算机辅助肺癌诊断方法[J].计算机辅助设计与图形学学报,2018,30(1):90-99)
[3]Liu W P,Otake Y,Azizian M,et al.2D-3D radiograph to cone-beam computed tomography(CBCT)registration for C-arm image-guided robotic surgery[J].International Journal of Computer Assisted Radiology and Surgery,2015,10(8):1239-1252
[4]Markelj P,Tomazevic D,Pernus F,et al.Robust gradient-based3-D/2-D registration of CT and MR to X-ray images[J].IEEETransactions on Medical Imaging,2008,27(12):1704-1714
[5]Lootus M,Kadir T,Zisserman A.Vertebrae detection and labelling in lumbar MR images[C]//Proceedings of Computational Methods and Clinical Applications for Spine Imaging.Heidelberg:Springer,2014,17:219-230
[6]Klinder T,Ostermann J,Ehm M,et al.Automated model-based vertebra detection,identification,and segmentation in CT images[J].Medical Image Analysis,2009,13(3):471-482
[7]Rasoulian A,Rohling R N,Abolmaesumi P.Automatic labeling and segmentation of vertebrae in CT images[C]//Proceedings of SPIE.Bellingham:Society of Photo-Optical Instrumentation Engineers Press,2014,9036:Article No.903623
[8]Glocker B,Feulner J,Criminisi A,et al.Automatic localization and identification of vertebrae in arbitrary field-of-view CTscans[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer,2012,7512:590-598
[9]Glocker B,Zikic D,Konukoglu E,et al.Vertebrae localization in pathological spine CT via dense classification from sparse annotations[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer,2013,8150:262-270
[10]Richmond D,Kainmueller D,Glocker B,et al.Uncertaintydriven forest predictors for vertebra localization and segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer,2015,9349:653-660
[11]Kadoury S,Labelle H,Paragios N.Automatic inference of articulated spine models in CT images using high-order Markov random fields[J].Medical Image Analysis,2011,15(4):426-437
[12]Ma J,Lu L.Hierarchical segmentation and identification of thoracic vertebra using learning-based edge detection and coarseto-fine deformable model[J].Computer Vision and Image Understanding,2013,117(9):1072-1083
[13]Cai Y L,Osman S,Sharma M,et al.Multi-modality vertebra recognition in arbitrary views using 3D deformable hierarchical model[J].IEEE Transactions on Medical Imaging,2015,34(8):1676-1693
[14]Yang D,Xiong T,Xu D G,et al.Deep image-to-image recurrent network with shape basis learning for automatic vertebra labeling in large-scale 3D CT volumes[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer,2017,10435:498-506
[15]Suzani A,Rasoulian A,Seitel A,et al.Deep learning for automatic localization,identification,and segmentation of vertebral bodies in volumetric MR images[C]//Proceedings of SPIE.Bellingham:Society of Photo-Optical Instrumentation Engineers Press,2015,9415:Article No.941514
[16]Yang D,Xiong T,Xu D G,et al.Automatic vertebra labeling in large-scale 3D CT using deep image-to-image network with message passing and sparsity regularization[C]//Proceedings of International Conference on Information Processing in Medical Imaging.Heidelberg:Springer,2017,10265:633-644
[17]Chen H,Shen C Y,Qin J,et al.Automatic localization and identification of vertebrae in spine CT via a joint learning model with deep neural networks[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer,2015,9349:515-522
[18]Cai Y L,Landis M,Laidley D T,et al.Multi-modal vertebrae recognition using transformed deep convolution network[J].Computerized Medical Imaging&Graphics,2016,51:11-19
[19]Chang C J,Lin G L,Tse A,et al.Registration of 2D C-arm and3D CT images for a C-arm image-assisted navigation system for spinal surgery[J].Applied Bionics and Biomechanics,2015,2015:Article No.478062
[20]Schuster M,Paliwal K K.Bidirectional recurrent neural networks[J].IEEE Transactions on Signal Processing,1997,45(1):2673-2681
[21]Du Y,Fu Y,Wang L.Representation learning of temporal dynamics for skeleton-based action recognition[J].IEEE Transactions on Image Processing,2016,25(7):3010-3022
[22]Li Y,Liang W,Tan J D,et al.A novel automatically initialized level set approach based on region correlation for lumbar vertebrae CT image segmentation[C]//Proceedings of the IEEEInternational Symposium on Medical Measurements and Applications.Los Alamitos:IEEE Computer Society Press,2015:291-296
[2]Zhang Peng,Xu Xinnan,Wang Hongwei,et al.Computer-aided lung cancer diagnosis approaches based on deep learning[J].Journal of Computer-Aided Design&Computer Graphics,2018,30(1):90-99(in Chinese)(张鹏,徐欣楠,王洪伟,等.基于深度学习的计算机辅助肺癌诊断方法[J].计算机辅助设计与图形学学报,2018,30(1):90-99)
[3]Liu W P,Otake Y,Azizian M,et al.2D-3D radiograph to cone-beam computed tomography(CBCT)registration for C-arm image-guided robotic surgery[J].International Journal of Computer Assisted Radiology and Surgery,2015,10(8):1239-1252
[4]Markelj P,Tomazevic D,Pernus F,et al.Robust gradient-based3-D/2-D registration of CT and MR to X-ray images[J].IEEETransactions on Medical Imaging,2008,27(12):1704-1714
[5]Lootus M,Kadir T,Zisserman A.Vertebrae detection and labelling in lumbar MR images[C]//Proceedings of Computational Methods and Clinical Applications for Spine Imaging.Heidelberg:Springer,2014,17:219-230
[6]Klinder T,Ostermann J,Ehm M,et al.Automated model-based vertebra detection,identification,and segmentation in CT images[J].Medical Image Analysis,2009,13(3):471-482
[7]Rasoulian A,Rohling R N,Abolmaesumi P.Automatic labeling and segmentation of vertebrae in CT images[C]//Proceedings of SPIE.Bellingham:Society of Photo-Optical Instrumentation Engineers Press,2014,9036:Article No.903623
[8]Glocker B,Feulner J,Criminisi A,et al.Automatic localization and identification of vertebrae in arbitrary field-of-view CTscans[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer,2012,7512:590-598
[9]Glocker B,Zikic D,Konukoglu E,et al.Vertebrae localization in pathological spine CT via dense classification from sparse annotations[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer,2013,8150:262-270
[10]Richmond D,Kainmueller D,Glocker B,et al.Uncertaintydriven forest predictors for vertebra localization and segmentation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer,2015,9349:653-660
[11]Kadoury S,Labelle H,Paragios N.Automatic inference of articulated spine models in CT images using high-order Markov random fields[J].Medical Image Analysis,2011,15(4):426-437
[12]Ma J,Lu L.Hierarchical segmentation and identification of thoracic vertebra using learning-based edge detection and coarseto-fine deformable model[J].Computer Vision and Image Understanding,2013,117(9):1072-1083
[13]Cai Y L,Osman S,Sharma M,et al.Multi-modality vertebra recognition in arbitrary views using 3D deformable hierarchical model[J].IEEE Transactions on Medical Imaging,2015,34(8):1676-1693
[14]Yang D,Xiong T,Xu D G,et al.Deep image-to-image recurrent network with shape basis learning for automatic vertebra labeling in large-scale 3D CT volumes[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer,2017,10435:498-506
[15]Suzani A,Rasoulian A,Seitel A,et al.Deep learning for automatic localization,identification,and segmentation of vertebral bodies in volumetric MR images[C]//Proceedings of SPIE.Bellingham:Society of Photo-Optical Instrumentation Engineers Press,2015,9415:Article No.941514
[16]Yang D,Xiong T,Xu D G,et al.Automatic vertebra labeling in large-scale 3D CT using deep image-to-image network with message passing and sparsity regularization[C]//Proceedings of International Conference on Information Processing in Medical Imaging.Heidelberg:Springer,2017,10265:633-644
[17]Chen H,Shen C Y,Qin J,et al.Automatic localization and identification of vertebrae in spine CT via a joint learning model with deep neural networks[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention.Heidelberg:Springer,2015,9349:515-522
[18]Cai Y L,Landis M,Laidley D T,et al.Multi-modal vertebrae recognition using transformed deep convolution network[J].Computerized Medical Imaging&Graphics,2016,51:11-19
[19]Chang C J,Lin G L,Tse A,et al.Registration of 2D C-arm and3D CT images for a C-arm image-assisted navigation system for spinal surgery[J].Applied Bionics and Biomechanics,2015,2015:Article No.478062
[20]Schuster M,Paliwal K K.Bidirectional recurrent neural networks[J].IEEE Transactions on Signal Processing,1997,45(1):2673-2681
[21]Du Y,Fu Y,Wang L.Representation learning of temporal dynamics for skeleton-based action recognition[J].IEEE Transactions on Image Processing,2016,25(7):3010-3022
[22]Li Y,Liang W,Tan J D,et al.A novel automatically initialized level set approach based on region correlation for lumbar vertebrae CT image segmentation[C]//Proceedings of the IEEEInternational Symposium on Medical Measurements and Applications.Los Alamitos:IEEE Computer Society Press,2015:291-296