融合进高层语义特征的医学图像检索技术研究
|
摘要
近年来,随着医学成像技术的迅猛发展和应用,医学图像在临床诊断、治疗以及研究方面起着越来越重要的作用。基于内容的医学图像检索技术是从海量的医学图像中检索出具有相似病理特性图像的技术。由于医学图像具有灰度及空间分辨率高,信息量大,图片比较相似的特点,因此基于内容的医学图像检索往往比一般的图像检索要复杂。
本研究针对胸片图像、胸部CT扫描图像和胃窥镜图像的融合进高层语义特征的检索技术进行了较深入的探讨,包括系统的体系结构、医学图像底层特征的提取、高层语义特征的提取、综合检索、相关反馈等。其中医学图像底层特征的提取主要基于颜色、纹理、形状、空间关系四大类,高层语义特征的提取主要基于医生诊断关键词,基于底层特征的潜在语义索引技术以及基于感兴趣区域的目标语义提取技术。
对于胸片图像,医生的诊断信息比较全面,因此提出一种针对胸片图像的检索方法。该方法提取医生诊断信息的关键字,以关键字的出现频率作为语义特征向量,再结合底层特征以形成检索特征向量进行检索。对于彩色胃窥镜图像,根据其本身丰富的色彩信息,提出一种针对胃窥镜图像的检索方法。该方法对多种典型的颜色及空间关系提取算法进行了比较分析,选出最适合做进一步语义提取的颜色直方图法和颜色自相关图法,在此基础上利用潜在语义索引技术,提取潜在语义,去除噪声,以期达到提高检索效率的目的。对于胸部CT图像,医生的观察焦点主要在病灶区域,因此提出一种针对胸部CT图像的检索方法。该方法先提取感兴趣目标语义区域,然后在此感兴趣区域的基础上提取视觉特征,最后利用这些目标语义区域上的底层特征来检索图像。
在以上思想的基础上,本研究设计了一个原型系统。对于本研究建立的医学图像库,利用不同的特征向量进行检索,并对各种算法的实验结果进行了比较分析。
In recent years, medical image is getting more and more important effect in the clinical diagnosis, curing and the aspect of researching with the fact that medical imaging technology swiftness developing and applying. The content-based medical image retrieval is a technology to retrieve the image with similar pathological mechanism property from great quantity of medical images. Since the medical image has the characteristics of high gradation and the space resolution ratio, big information amounts and great similarity, the content-based medical image retrieval sometimes is more complicated than the ordinary image retrieval.
Our work has a discussion as to sternum image, gastroscope image and chest CT image’s retrieval technology which fused with high-level semantic features, including system structure, low-level feature extraction, high-level semantic feature extraction, integrated retrieval and relevance feedback. Low-level feature extraction is based on color, texture, shape and spatial information of medical image. High-level semantic feature extraction is based on doctor’s diagnosis keywords, latent semantic indexing technology and object semantic extraction technology based on regions of interest.
As to sternum image, the doctor’s diagnosis information is relatively comprehensive. A method for sternum image retrieval is proposed. The method extract the key words of doctor’s diagnosis information, and use the occurrence frequency of key words as semantic feature vector, and then combine low-level features to construct retrieval feature vector to retrieval. As to colorful gastroscope image, according to its own colorful information, a method for gastroscope image retrieval is proposed. We analyze several typical color and spatial information extraction algorithm, and then select the most adapt to semantic extraction algorithm, make use of latent semantic indexing technology to remove noise in order to improve the retrieval performance. As to chest CT image, doctor’s focus is on the focus area of the image. A method for chest CT image retrieval is proposed. We extract the region of interest, and then extract visual feature upon the region of interest, at last use the feature which based on the target semantic region to retrieve image.
Based on above algorithms, our research designed a prototype system. As to the image database established by the research, we use different feature vectors to retrieve and several experiments are made to compare the above algorithms.
本研究针对胸片图像、胸部CT扫描图像和胃窥镜图像的融合进高层语义特征的检索技术进行了较深入的探讨,包括系统的体系结构、医学图像底层特征的提取、高层语义特征的提取、综合检索、相关反馈等。其中医学图像底层特征的提取主要基于颜色、纹理、形状、空间关系四大类,高层语义特征的提取主要基于医生诊断关键词,基于底层特征的潜在语义索引技术以及基于感兴趣区域的目标语义提取技术。
对于胸片图像,医生的诊断信息比较全面,因此提出一种针对胸片图像的检索方法。该方法提取医生诊断信息的关键字,以关键字的出现频率作为语义特征向量,再结合底层特征以形成检索特征向量进行检索。对于彩色胃窥镜图像,根据其本身丰富的色彩信息,提出一种针对胃窥镜图像的检索方法。该方法对多种典型的颜色及空间关系提取算法进行了比较分析,选出最适合做进一步语义提取的颜色直方图法和颜色自相关图法,在此基础上利用潜在语义索引技术,提取潜在语义,去除噪声,以期达到提高检索效率的目的。对于胸部CT图像,医生的观察焦点主要在病灶区域,因此提出一种针对胸部CT图像的检索方法。该方法先提取感兴趣目标语义区域,然后在此感兴趣区域的基础上提取视觉特征,最后利用这些目标语义区域上的底层特征来检索图像。
在以上思想的基础上,本研究设计了一个原型系统。对于本研究建立的医学图像库,利用不同的特征向量进行检索,并对各种算法的实验结果进行了比较分析。
In recent years, medical image is getting more and more important effect in the clinical diagnosis, curing and the aspect of researching with the fact that medical imaging technology swiftness developing and applying. The content-based medical image retrieval is a technology to retrieve the image with similar pathological mechanism property from great quantity of medical images. Since the medical image has the characteristics of high gradation and the space resolution ratio, big information amounts and great similarity, the content-based medical image retrieval sometimes is more complicated than the ordinary image retrieval.
Our work has a discussion as to sternum image, gastroscope image and chest CT image’s retrieval technology which fused with high-level semantic features, including system structure, low-level feature extraction, high-level semantic feature extraction, integrated retrieval and relevance feedback. Low-level feature extraction is based on color, texture, shape and spatial information of medical image. High-level semantic feature extraction is based on doctor’s diagnosis keywords, latent semantic indexing technology and object semantic extraction technology based on regions of interest.
As to sternum image, the doctor’s diagnosis information is relatively comprehensive. A method for sternum image retrieval is proposed. The method extract the key words of doctor’s diagnosis information, and use the occurrence frequency of key words as semantic feature vector, and then combine low-level features to construct retrieval feature vector to retrieval. As to colorful gastroscope image, according to its own colorful information, a method for gastroscope image retrieval is proposed. We analyze several typical color and spatial information extraction algorithm, and then select the most adapt to semantic extraction algorithm, make use of latent semantic indexing technology to remove noise in order to improve the retrieval performance. As to chest CT image, doctor’s focus is on the focus area of the image. A method for chest CT image retrieval is proposed. We extract the region of interest, and then extract visual feature upon the region of interest, at last use the feature which based on the target semantic region to retrieve image.
Based on above algorithms, our research designed a prototype system. As to the image database established by the research, we use different feature vectors to retrieve and several experiments are made to compare the above algorithms.
引文
[1]周杰.医学图像特征的自动获取与基于内容检索的方法研究:博士学位论文.第一军医大学,2004
[2] D. T. Hemant, M. Frans, C. Conrade, S. James, et al. Arrangement: A Spatial Relation Between Parts for Evaluating Similarity of Tomographic Section. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995, 17 (9):880-893
[3] P. M. Kelly, M. Cannon, J. E. Barros. Efficiency issues related to probability density function comparison. In: Ishwar K. Sethi, Ramesh Jain eds. SPIE Proceedings, Storage and Retrieval for Image and Video Databases IV, SPIE Proceedings Vol. 2670, San Diego/La Jolla, CA, USA,1996:IEEE, 1996, 42–49
[4] C. T. Liu, P. L. Tai, A. Y. Chen, et al. A content based medical teaching file assistant for CT lung image retrieval. In: Proceedings of the IEEE International Conference on Electronics,Circuits, Systems (ICECS2000), Jouneih Kaslik, Lebanon, 2000
[5] D. Comaniciu, P. Meer, D. Foran, et al. Bimodal system for interactive indexing and retrieval of pathology images. In: Proceedings of the Fourth IEEE Workshop on Applications of Computer Vision (WACV'98), Princeton, NJ, USA, 1998, 76-81
[6] W. J. Kuo, R. F. Chang, C. C. Lee, et al. Retrieval technique for the diagnosis of solid breast tumors on sonogram. Ultrasound in Medicine and Biology, 2002, 28(7):903-909
[7] Weidong Cai, Dagan Feng, R. Fulton. Content-based retrieval of dynamic PET functional images. IEEE Transactions on Information Technology in Biomedicine, 2000, 4(2):152-158
[8]赵晨光,庄天戈.基于内容的医学图像检索.国外医学生物医学工程分册,2004, 27(2):83-86
[9]金丰华,罗立民.基于内容的医学图像检索.东南大学硕士学位论文,2003
[10]夏顺仁,莫伟荣,王小英,严勇.基于特征融合和相关反馈的医学图像检索技术.航天医学与医学工程,2004,17(6):429～433
[11]邵虹,张继武,崔文成,赵宏.一种基于形状特征的颅骨图像检索方法.计算机工程,2003,29(8):1-4
[12] Shunshan Li, Tiange Zhuang, Hui Chen. Medical image retrieval based on mutual correlation method. SPIE, 2001, Vol.4323
[13]杨敏.基于内容的医学图像检索中特征提取技术研究:硕士学位论文.西北工业大学,2007
[14]黄祥林,沈兰荪.基于内容的图像检索技术研究.电子学报,2002, 30(7): 1065~1071
[15] Rafael C. Gonzalez, Richard E. Woods. Digital Image Proceeding. 2nd ed.北京:电子工业出版社,2003
[16] SMITH JR. Intergrated Spatialand feature image systems retrieval analysis and compression[D]. Graduate school of arts and sciences, Columnbia university, 1997
[17]邰晓英,北研二.信息检索技术导论.北京:科学出版社, 2006
[18] M J Swain, D H Ballard. Color indexing. International Journal of Computer Vision, 1991,7(1): 11~32
[19]林三仁.消化系统疾病电子内镜图谱.北京:北京大学医学出版社, 2003
[20]吴贤伟,邰晓英,巴特尔.基于内容的彩色胃镜图像检索.计算机应用,2005, 25(12): 248~250
[21] Markus Stricker, Markus Orengo. Similarity of Color Images. In Proc. of SPIE Storage and Retrieval for Image and Video Databases III. San Jose, CA: SPIE Press, 1995. 381~392
[22]黄朝兵,余胜生,周敬利,鲁宏伟.基于多邻域统计矩直方图的彩色图像检索.小型微型计算机系统, 2005.26(6):1061~1064
[23] HUANG J, KUMAR SR, MITRA M, et al. Image indexing using color correlograms. Computer Vision Pattern Recognition, 1997:762~768
[24] Wallace TM.胸部X线阅片指南.天津科技翻译出版公司, 2001
[25]胡小锋,赵辉. Visual C++/MATLAB图像处理与识别实用案例精选.北京:人民邮电出版社, 2004
[26] Haralick R.M. Statistical and structural approaches to texture. Proceedings of IEEE, 67(5): 786~804
[27] Ortega M, et al. Supporting similarity queries in MARS. Proc ACM Multimedia, 403~413
[28]洪继光.灰度—梯度共生矩阵纹理分析方法.自动化学报,1984, 10(1): 22~25
[29]高守传,姚领田等. Visual C++实践与提高——数字图像处理与工程应用篇.北京:中国铁道出版社,2006
[30]庄越挺,潘云鹤,吴飞.网上多媒体信息分析与检索.北京:清华大学出版社, 2002
[31] Teague M R. Image analysis via the general theory of moments. Opt Soc Am, 1980,70 (8) : 920～930
[32] A. Khotanzad, C. Chung. Application of Multi-Layer Perceptron Neural Networks to Vision Problems. Neural Comput & Applic, 1998, 7: 249~259
[33] Javad Haddadnia, Majid Ahmadi and Karim Faez. Human Face Recognition with Different Statistical Features. Lecture Notes in Computer Science, Structural, Syntactic, and Statistical Pattern Recognition. Springer Berlin, 2002
[34]施智平,高光来,赵晓春,苏林英.一个基于颜色特征的图像检索.内蒙古大学学报,2003, 34(2): 226~230
[35] Huang J, Kumar SR, Mitra M, et al. Spatial color indexing and applications. International Journal of Computer Vision, 1999:245~268
[36] Smith LI. A tutorial on principal. Components Analysis, 2002.2
[37]周长发.科学与工程数值算法.北京:清华大学出版社, 2002
[38]王惠锋,金翔宇,孙正兴.基于语义的图像检索方法研究综述.计算机工程,2001, 27(8): 3~6
[39]邵虹.基于内容的医学图像检索关键技术研究:博士学位论文.东北大学,2004
[40] Zhao R, Grosky W I. Negotiating the semantic gap: From feature maps to semantic landscapes. Pattern Recognition, 2002, 35: 593~600
[41]袁磊,曹奎,冯玉才等.一种基于LSI的图像语义检索技术.华中科技大学学报(自然科学版), 2002.30(2): 105~107
[42] S.T. Dumais. Improving the retrieval of information from external sources. Behavior Research Methods. Instruments & Computers, 1991, 23(2): 229~236
[43]周水庚,关佶红,胡运发.隐含语义索引及其在中文文本处理中的应用研究.小型微型计算机系统,2001, 22(2): 239~243
[44]章毓晋.图像分割.北京:科学出版社,2001
[45]田捷,包尚联,周明全.医学影像处理与分析.北京:电子工业出版社, 2002,9
[46]何晖光,田捷,赵明昌,杨骅.基于分割的三维医学图像表面重建算法.软件学报, 2002,13(2):219～226
[47] Milan Sonka, Vaclav Hlavac, and roger Boyle. Image Processing, Analysis, and Machine Vision, pages 123-227. Brooks/Cole, a division of Thomson Asia Pte Led, 2nd ed. 2000
[48] Kenneth R. and Castleman. Digital Image Processing, pages 447-486. Prentice-Hall, 1979
[49] S.Z.Li. Markov Random Field Modeling in Computer Vision. Springer, 1995
[50] Stuart Geman and Donald Geman. Stochastic relaxation gibbs distribution, and the Bayesian restoration of images. IEEE transactions on Pattern Analysis and Machine Intelligence, 6(6), 1984
[51] Michael Kass, Andrew Witkin, and Demetri Terzopoulos. Snake: Active contour models. In First International Conference of Computer Vision, pages 259-268, London, 1987
[52] Laurent D.Cohen. On active contour models and ballons. Computer Vision Graphics Image Processing: Image Understanding, 53: 211-218, 1991
[53] Frederic Leymarie and Martin D. Levine. Tracking deformable objects in the plane using an active contour model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15(6): 617-634
[54] J.A. Sethian. Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision and Materials Science. Cambridge University Press, second edition, 1999
[55]唐立军.基于内容的图像检索系统研究和设计:硕士学位论文.中国科学院研究生院(计算技术研究所),2001
[56]周建新,高科,李锦涛,张勇东,唐胜.图像检索中一种有效的SVM相关反馈算法.计算机辅助设计与图形学学报,2007,19(4): 535-540
[57]何京芮.图像检索中的统计学习方法研究:硕士学位论文.清华大学,2005
[58]王李东.基于内容的医学图像检索算法研究:硕士学位论文.宁波大学,2006
[2] D. T. Hemant, M. Frans, C. Conrade, S. James, et al. Arrangement: A Spatial Relation Between Parts for Evaluating Similarity of Tomographic Section. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995, 17 (9):880-893
[3] P. M. Kelly, M. Cannon, J. E. Barros. Efficiency issues related to probability density function comparison. In: Ishwar K. Sethi, Ramesh Jain eds. SPIE Proceedings, Storage and Retrieval for Image and Video Databases IV, SPIE Proceedings Vol. 2670, San Diego/La Jolla, CA, USA,1996:IEEE, 1996, 42–49
[4] C. T. Liu, P. L. Tai, A. Y. Chen, et al. A content based medical teaching file assistant for CT lung image retrieval. In: Proceedings of the IEEE International Conference on Electronics,Circuits, Systems (ICECS2000), Jouneih Kaslik, Lebanon, 2000
[5] D. Comaniciu, P. Meer, D. Foran, et al. Bimodal system for interactive indexing and retrieval of pathology images. In: Proceedings of the Fourth IEEE Workshop on Applications of Computer Vision (WACV'98), Princeton, NJ, USA, 1998, 76-81
[6] W. J. Kuo, R. F. Chang, C. C. Lee, et al. Retrieval technique for the diagnosis of solid breast tumors on sonogram. Ultrasound in Medicine and Biology, 2002, 28(7):903-909
[7] Weidong Cai, Dagan Feng, R. Fulton. Content-based retrieval of dynamic PET functional images. IEEE Transactions on Information Technology in Biomedicine, 2000, 4(2):152-158
[8]赵晨光,庄天戈.基于内容的医学图像检索.国外医学生物医学工程分册,2004, 27(2):83-86
[9]金丰华,罗立民.基于内容的医学图像检索.东南大学硕士学位论文,2003
[10]夏顺仁,莫伟荣,王小英,严勇.基于特征融合和相关反馈的医学图像检索技术.航天医学与医学工程,2004,17(6):429～433
[11]邵虹,张继武,崔文成,赵宏.一种基于形状特征的颅骨图像检索方法.计算机工程,2003,29(8):1-4
[12] Shunshan Li, Tiange Zhuang, Hui Chen. Medical image retrieval based on mutual correlation method. SPIE, 2001, Vol.4323
[13]杨敏.基于内容的医学图像检索中特征提取技术研究:硕士学位论文.西北工业大学,2007
[14]黄祥林,沈兰荪.基于内容的图像检索技术研究.电子学报,2002, 30(7): 1065~1071
[15] Rafael C. Gonzalez, Richard E. Woods. Digital Image Proceeding. 2nd ed.北京:电子工业出版社,2003
[16] SMITH JR. Intergrated Spatialand feature image systems retrieval analysis and compression[D]. Graduate school of arts and sciences, Columnbia university, 1997
[17]邰晓英,北研二.信息检索技术导论.北京:科学出版社, 2006
[18] M J Swain, D H Ballard. Color indexing. International Journal of Computer Vision, 1991,7(1): 11~32
[19]林三仁.消化系统疾病电子内镜图谱.北京:北京大学医学出版社, 2003
[20]吴贤伟,邰晓英,巴特尔.基于内容的彩色胃镜图像检索.计算机应用,2005, 25(12): 248~250
[21] Markus Stricker, Markus Orengo. Similarity of Color Images. In Proc. of SPIE Storage and Retrieval for Image and Video Databases III. San Jose, CA: SPIE Press, 1995. 381~392
[22]黄朝兵,余胜生,周敬利,鲁宏伟.基于多邻域统计矩直方图的彩色图像检索.小型微型计算机系统, 2005.26(6):1061~1064
[23] HUANG J, KUMAR SR, MITRA M, et al. Image indexing using color correlograms. Computer Vision Pattern Recognition, 1997:762~768
[24] Wallace TM.胸部X线阅片指南.天津科技翻译出版公司, 2001
[25]胡小锋,赵辉. Visual C++/MATLAB图像处理与识别实用案例精选.北京:人民邮电出版社, 2004
[26] Haralick R.M. Statistical and structural approaches to texture. Proceedings of IEEE, 67(5): 786~804
[27] Ortega M, et al. Supporting similarity queries in MARS. Proc ACM Multimedia, 403~413
[28]洪继光.灰度—梯度共生矩阵纹理分析方法.自动化学报,1984, 10(1): 22~25
[29]高守传,姚领田等. Visual C++实践与提高——数字图像处理与工程应用篇.北京:中国铁道出版社,2006
[30]庄越挺,潘云鹤,吴飞.网上多媒体信息分析与检索.北京:清华大学出版社, 2002
[31] Teague M R. Image analysis via the general theory of moments. Opt Soc Am, 1980,70 (8) : 920～930
[32] A. Khotanzad, C. Chung. Application of Multi-Layer Perceptron Neural Networks to Vision Problems. Neural Comput & Applic, 1998, 7: 249~259
[33] Javad Haddadnia, Majid Ahmadi and Karim Faez. Human Face Recognition with Different Statistical Features. Lecture Notes in Computer Science, Structural, Syntactic, and Statistical Pattern Recognition. Springer Berlin, 2002
[34]施智平,高光来,赵晓春,苏林英.一个基于颜色特征的图像检索.内蒙古大学学报,2003, 34(2): 226~230
[35] Huang J, Kumar SR, Mitra M, et al. Spatial color indexing and applications. International Journal of Computer Vision, 1999:245~268
[36] Smith LI. A tutorial on principal. Components Analysis, 2002.2
[37]周长发.科学与工程数值算法.北京:清华大学出版社, 2002
[38]王惠锋,金翔宇,孙正兴.基于语义的图像检索方法研究综述.计算机工程,2001, 27(8): 3~6
[39]邵虹.基于内容的医学图像检索关键技术研究:博士学位论文.东北大学,2004
[40] Zhao R, Grosky W I. Negotiating the semantic gap: From feature maps to semantic landscapes. Pattern Recognition, 2002, 35: 593~600
[41]袁磊,曹奎,冯玉才等.一种基于LSI的图像语义检索技术.华中科技大学学报(自然科学版), 2002.30(2): 105~107
[42] S.T. Dumais. Improving the retrieval of information from external sources. Behavior Research Methods. Instruments & Computers, 1991, 23(2): 229~236
[43]周水庚,关佶红,胡运发.隐含语义索引及其在中文文本处理中的应用研究.小型微型计算机系统,2001, 22(2): 239~243
[44]章毓晋.图像分割.北京:科学出版社,2001
[45]田捷,包尚联,周明全.医学影像处理与分析.北京:电子工业出版社, 2002,9
[46]何晖光,田捷,赵明昌,杨骅.基于分割的三维医学图像表面重建算法.软件学报, 2002,13(2):219～226
[47] Milan Sonka, Vaclav Hlavac, and roger Boyle. Image Processing, Analysis, and Machine Vision, pages 123-227. Brooks/Cole, a division of Thomson Asia Pte Led, 2nd ed. 2000
[48] Kenneth R. and Castleman. Digital Image Processing, pages 447-486. Prentice-Hall, 1979
[49] S.Z.Li. Markov Random Field Modeling in Computer Vision. Springer, 1995
[50] Stuart Geman and Donald Geman. Stochastic relaxation gibbs distribution, and the Bayesian restoration of images. IEEE transactions on Pattern Analysis and Machine Intelligence, 6(6), 1984
[51] Michael Kass, Andrew Witkin, and Demetri Terzopoulos. Snake: Active contour models. In First International Conference of Computer Vision, pages 259-268, London, 1987
[52] Laurent D.Cohen. On active contour models and ballons. Computer Vision Graphics Image Processing: Image Understanding, 53: 211-218, 1991
[53] Frederic Leymarie and Martin D. Levine. Tracking deformable objects in the plane using an active contour model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15(6): 617-634
[54] J.A. Sethian. Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision and Materials Science. Cambridge University Press, second edition, 1999
[55]唐立军.基于内容的图像检索系统研究和设计:硕士学位论文.中国科学院研究生院(计算技术研究所),2001
[56]周建新,高科,李锦涛,张勇东,唐胜.图像检索中一种有效的SVM相关反馈算法.计算机辅助设计与图形学学报,2007,19(4): 535-540
[57]何京芮.图像检索中的统计学习方法研究:硕士学位论文.清华大学,2005
[58]王李东.基于内容的医学图像检索算法研究:硕士学位论文.宁波大学,2006