小波在视频检索中的应用
摘要
随着数字化图书馆和大型多媒体数据库的广泛使用。视频检索将成为重要的研究领域。最大程度地提高基于内容的多媒体的应用,也正是MPEG-7的目的所在。
小波分析是继Fourier分析之后新的时频分析工具,它在科学研究和工程技术中的应用非常广泛。虽然小波理论现已比较成熟,但是近年来有关它的应用研究仍在不断发展更新。小波变换在图像处理领域中的应用几乎可以囊括图像处理的所有方面。本文对小波变换在视频检索方面进行了研究,在分析了这些方面研究现状的基础上,探讨了自己的算法和思想,并给出了相应的实验结果和分析,以下是本文主要研究内容:
(1)由于视频序列可以看成帧内二维和时间上的一维,因此可以把已有的图像检索的方法应用到视频检索中。首先利用基于小波的颜色图像检索算法对视频库进行初步筛选,然后根据集合的相似性度量原理对查询片断和目标片断进行相似性的计算,相似度值最大的则表示查询片断和目标片断完全一样,同时该目标片断就是所要求查询的片断。通过实验分析,该方法不仅可以获得较高检索效率,而且有着更高的检索速度。
(2)从另一个角度来看,又可以把视频数据看一个三维信号,然后对其进行三维小波变换分解,该方法克服了传统视频检索方法中未考虑的时空关系。首先利用低频子图小波系数的标准方差均值对目标视频库进行初步筛选,然后利用八叉树算法提取细节特征,进一步的把细节特征和近似特征组合起来对图像进行精确的检索。实验证明,该方法具有更好的检索精度,但是由于视频数据量比较大,因此利用此方法在速度上将明显降低。
Video retrieval is emerging as an important research area with application to digital libraries and multimedia databases. The objective of the MPEG-7 standard is to enhance content-based multimedia applications to the maximum.
For another, wavelet transform is a new tool of time-frequency analysis after Fourier analysis and is widely applied in the scientific research and project technology. Though wavelet theory has been somewhat perfect, its applied research still has been developed and renovated. The image processing field where wavelet transform is applied almost includes the whole image processing aspects. Video retrieval has been researched in the paper. Based on the analysis in these research actualities, some algorithms and ideas have been proposed and the relevant test result and analysis have been given. The main research contents are as follows.
Video sequences are viewed as 2-D among the frames and 1-D of time, so image retrieval algorithms presented by others can be applied to video retrieval. First, the video libraries are filtered by image retrieval algorithm based on wavelet, and then calculate the similarity between query clip and target clip according to the similarity measure principle, the maximal similarity indicates that the target clip is the same as the query clip. The target clip is the one clip to be inquired. Experimental indicates that the proposed methods is not only efficient in retrieving the video clip, but also can gain the higher search speed.
From the other aspects, the video data can also be regarded as three -dimensional signal, and is decomposed by 3-D Wavelet Transform. This way overcomes the relations between temporal and spatial. First, the target video library is filtered by the average of wavelet -coefficient standard deviation of the approximation sub-image. Then, the detail feature is extracted using the octree algorithm. Finally, the video clip was retrieved accurately by combining the detail features with the approximate features. Experimental results indicated that the proposed approach is efficient in retrieving video clips.
小波分析是继Fourier分析之后新的时频分析工具,它在科学研究和工程技术中的应用非常广泛。虽然小波理论现已比较成熟,但是近年来有关它的应用研究仍在不断发展更新。小波变换在图像处理领域中的应用几乎可以囊括图像处理的所有方面。本文对小波变换在视频检索方面进行了研究,在分析了这些方面研究现状的基础上,探讨了自己的算法和思想,并给出了相应的实验结果和分析,以下是本文主要研究内容:
(1)由于视频序列可以看成帧内二维和时间上的一维,因此可以把已有的图像检索的方法应用到视频检索中。首先利用基于小波的颜色图像检索算法对视频库进行初步筛选,然后根据集合的相似性度量原理对查询片断和目标片断进行相似性的计算,相似度值最大的则表示查询片断和目标片断完全一样,同时该目标片断就是所要求查询的片断。通过实验分析,该方法不仅可以获得较高检索效率,而且有着更高的检索速度。
(2)从另一个角度来看,又可以把视频数据看一个三维信号,然后对其进行三维小波变换分解,该方法克服了传统视频检索方法中未考虑的时空关系。首先利用低频子图小波系数的标准方差均值对目标视频库进行初步筛选,然后利用八叉树算法提取细节特征,进一步的把细节特征和近似特征组合起来对图像进行精确的检索。实验证明,该方法具有更好的检索精度,但是由于视频数据量比较大,因此利用此方法在速度上将明显降低。
Video retrieval is emerging as an important research area with application to digital libraries and multimedia databases. The objective of the MPEG-7 standard is to enhance content-based multimedia applications to the maximum.
For another, wavelet transform is a new tool of time-frequency analysis after Fourier analysis and is widely applied in the scientific research and project technology. Though wavelet theory has been somewhat perfect, its applied research still has been developed and renovated. The image processing field where wavelet transform is applied almost includes the whole image processing aspects. Video retrieval has been researched in the paper. Based on the analysis in these research actualities, some algorithms and ideas have been proposed and the relevant test result and analysis have been given. The main research contents are as follows.
Video sequences are viewed as 2-D among the frames and 1-D of time, so image retrieval algorithms presented by others can be applied to video retrieval. First, the video libraries are filtered by image retrieval algorithm based on wavelet, and then calculate the similarity between query clip and target clip according to the similarity measure principle, the maximal similarity indicates that the target clip is the same as the query clip. The target clip is the one clip to be inquired. Experimental indicates that the proposed methods is not only efficient in retrieving the video clip, but also can gain the higher search speed.
From the other aspects, the video data can also be regarded as three -dimensional signal, and is decomposed by 3-D Wavelet Transform. This way overcomes the relations between temporal and spatial. First, the target video library is filtered by the average of wavelet -coefficient standard deviation of the approximation sub-image. Then, the detail feature is extracted using the octree algorithm. Finally, the video clip was retrieved accurately by combining the detail features with the approximate features. Experimental results indicated that the proposed approach is efficient in retrieving video clips.
引文
[1] Panchanathan.S,Mandal.M.K, Video indexing in the wavelet compressed domain[R].Image Processing,1998.ICIP 98. Proceedings.1998 International Conference on 4-7 Oct.1998 Pages(s):546-550 vol.3
[2] Sen-ching S.Cheung,Avideh Zakhor,Fast Similarity Search and Clustering of Video Sequences on the World-Wide-Web[M].Multimeida,IEEE Transactions on Volume 7,Issue3,June 2005 Page(s):524-537
[3] Haas.M,Oerlemans.A,Lew.M.S.Relevance Feedback Methods in Content Based Retrieval and Video Summarization[R].Multimedia and Expo.2005.ICME2005.IEEE International Conference on 06-06 July 2005 Page(s):1038-1041
[4] Ngo CW, Pong TC, Zhang HJ. Motion-Based video representation for scene change detection[J]. International In: Proceedings of the ACM Multimedia,2000.
[5]施智平,李清勇,史俊.基于关键帧序列的视频片断检索[J].计算机应用.2005年Vol 25 No.8
[6]刘贵中等,小波分析及其应[M],西安电子科技大学出版社,1992。
[7]陈付华。小波在图像分析中的若干关键问题研究,南京理工大学博士学位论文,2002。
[8] YU-TE WU,TAKEO KANADE,CHING-CHUNG LI,JEFFREY COHN. Image Registration Using Wavelet-Based Motion Model. International Journal of Computer Vision 38(2), 129 -152 ,2000.
[9] Dimitrova N, Abdel-Mottaled M. Content-Based video retrieval by example video clip. In: Proceedings of IS&T and SPIE Storage and Retrieval of Image and Video Databases VI, Vol.3022. 1998. 184～196.
[10] Jain AK, Vailaya A, Wei X. Query by video clip. ACM Multimedia Systems, 1999, 7(5):369～384.
[11] Liu XM, Zhuang YT, Pan YH. A new approach to retrieve video by example video clip.In: Proceedings of ACM Multimedia.1999.41～44.
[12]李水根,吴纪桃,分形与小波,科学版,2002,pp196-316
[13]阮秋琦,数字图像处理学,电子工业版,2001, pp122-179
[14]陈武凡主编《.小波分析及其在图像处理中的应用》[M].北京:科学出出版社,2002。
[15]庄越挺,潘云鹤,吴飞编著.《网上多媒体信息分析与检索》[M].北京:清华大学出版社,2002.
[16] Wu Y, Zhuang YT, Pan YH. Content-Based video similarity model. In: Proceedings of the ACM Multimedia,2000.
[17] Zhuang YT, Liu XM, Wu Y, Pan YH. A new approach to retrieve video by example video clip. Chinese Journal of Computers, 2000,23(3):300～305.
[18] Chen LP, Chua TS. A match and tiling approach to content-based video retrieval. In: Proceedings of IEEE International Conference on Multimedia and Expo(ICME2001).2001.417～420.
[19] Orkun Alatas ,omar javed Mubarak Shah. Compressed Spatio-temporal Descriptors for video matching and retrieval.17th International Conference on Pattern Recognition (ICPR’04).Volume 3.
[20] Zhang H J, Wu Jianhua,. An Integrated System for Content-based Video Retrieval and Browsing[J ] . Pattern Recognition ,1997,Vol.30(4) page(s):643-657.
[21] Wolf Wayne. "Key frame selection by motion analysis". Proc. of IEEE Int. Conf. On Acoustics, Speech and Signal Processing, ICASSP, Atlanta,1996,page(s):7-10.
[22] Tekalp A Murat. Digital Video Processing[M] .北京:清华大学出版社,1998.
[23]彭宇新,Ngo Chong-Wah,董庆杰等.一种通过视频片段进行视频检索的方法.软件学报.2003.14(8).1409～1417
[24] Tong Lin ,et al. Video Content Representation for Shot Retrieval and Scene Extraction [J] . International Journal of Image and Graphics ,2001 ,1(3)
[25] Myung H,Senong K.A Multiplierless 2-D Convolver Chip for Real-Time Image Processing[J].Journal of VLSI Signal Processing,2004,38:page(s)63-71.
[26] Roy Utpal , Xu Yaoxian. Computation of a geometric model of a machined part from its NC machining programs[J ] . Computer-Aided Design ,1999 ,31 :401—411.
[27]宋传鸣,王相海.小波域视频运动估计研究进展.计算机学报.Vol.28 No.10,Oct.2005
[28] Cui Su-Xia. Motion estimation and compensation in the redundant wavelet domain [ Ph. D. dissertation ] . Mississippi State University , Mississippi State , USA , 2003
[29] J. Fridrich, M. Goljan, and D. Hogea, New methodology for breaking steganographic techniques for JPEGs,in SPIE Symposium on Electronic Imaging, (Santa Clara, CA), 2003
[30] Arredondo M A, lebart K, Lane D. Optical flow using textures[J]. Pattern Recognition Letters, 2004;25:449-457.
[31] Changbo Hu, Yi Li, Songde Ma et al. Region based parametric motion representation[J]. Pattern Recognition,2000,3:861-864.
[32] Vicente Grau, Mariano A R, Hierarchical image segmentation using a correspondence with a tree model[J]. Pattern Recognition, 2004;37:47-59.
[33]胡汉平,李德华,熊联欢等.基于小波变换的实时图像采集系统[J].数据采集与处理,1997,12(4):268-271.
[34]杨维明,周明,黄秋安. FFT专用芯片的应用[J].半导体技术,2002,27(2):54-56.
[35] Emanuel Radoi, Brigitte Hoeltzener. Improving the Radar Target Classification Results by Decision Fusion[C]. In: Proceedings of the International Radar Conference 2003,2003:162-165.
[36] Ganesan S A L. Texture classification using wavelet transform[J]. Pattern Recognition Letters, 2003,1(24):1513-1521.
[37] Ohm J R. A set of visual feature descriptors and their combination in a low-level description scheme[J]. Signal Processing: Image Communication,2000,9(16):157-179.
[38]黄卓君,马争鸣.多小波图像变换的统计分析[J].中国图像图形学报,2001,12(6):1198-2004.
[39] Tan YP, Kulkarni SR, Ramadge PJ. A framework for measuring video similarity and its application to video query by example .In: Proceedings of IEEE International Conference on Image Processing (ICIP 1999). 1999:106-110.
[40] Dimitrova N, Abdel-Mottaled M. Content-Based video retrieval by example video clip. In: Proceedings of IS&T and SPIE Storage and Retrieval of Image and Video Databases VI, Vol.3022.1998.184-196.
[41]林通,张宏江,封举富,石青云.镜头内容分析及其在视频检索中的应用.软件学报,2002,13(8):1577-1585.
[42]赵黎,祁卫,李子青,杨士强,张宏江.利用改进NFL算法对镜头进行基于内容的检索.软件学报,2002,13(4):586-590.
[43] Ekin A, Tekalp A M. Shot type classification by dominant color for sports video segmentation and summarization[A]. IEEE International Conference on Acoustics ,Speech, and Signal Processing 2003[C]. Hong Kong: IEEE Signal Processing Society, 2003.6-10.
[44] A Akharraz, Mauris G.A project Decision Support System Based on an Elucidative Fusion System[C]. In:Proceeding of the Fifth International Conference on Information Fusion 2002,2002-07:593-599.
[45] Jeannin S, Mory B. Video Motion Representation for Improved Content Access. IEEE Transactions on Consumer Electronics,2000,46(3).
[46] Ye Jiamin Smeaton A F. Aggregated Feature Retrieval for MPEG-7. ECIR 2003,LNCS 2633,2003:563-570.
[47] Jing H, Zhang H J, Video Segmentation with the Support of Audio Segmentation and Classification[J]. Proceedings of the IEEE Int. Conf. on Multimedia and Expo(ICME 2000),2000,3:1507-1510
[48]庄越挺.通过例子视频进行视频检索的新方法[J].计算机学报.2000,23(3):300-305.
[49] Meng J et al. Scene change detection in a MPEG compressed video sequence[C]. In:Proceedings of IS & T/SPIE International Symposium on Electronic Imaging, San Jose,1995:14-25.
[50] Frarg W, Abdel- Wahab H. A new paradigm for detecting scene changes on MPEG compressed videos[C], In: Proceedings of IEEE International Symposium on Signal Processing and Information Technology, Cairo, Egypt,2001:153-158.
[51] Change Sh2F, et al. VideoQ: An Automated Content-based Video Search System using Visual Cues[C]. Proc. Of ACM Multimedia’97.Seattle Washington USA, 1997.3132324
[52] Patke Nilesh V, Sethi lshwar K, Video shot detection and characterization for video databases, Pattern Recognition,vol 30,no 4,1997,583-592.
[2] Sen-ching S.Cheung,Avideh Zakhor,Fast Similarity Search and Clustering of Video Sequences on the World-Wide-Web[M].Multimeida,IEEE Transactions on Volume 7,Issue3,June 2005 Page(s):524-537
[3] Haas.M,Oerlemans.A,Lew.M.S.Relevance Feedback Methods in Content Based Retrieval and Video Summarization[R].Multimedia and Expo.2005.ICME2005.IEEE International Conference on 06-06 July 2005 Page(s):1038-1041
[4] Ngo CW, Pong TC, Zhang HJ. Motion-Based video representation for scene change detection[J]. International In: Proceedings of the ACM Multimedia,2000.
[5]施智平,李清勇,史俊.基于关键帧序列的视频片断检索[J].计算机应用.2005年Vol 25 No.8
[6]刘贵中等,小波分析及其应[M],西安电子科技大学出版社,1992。
[7]陈付华。小波在图像分析中的若干关键问题研究,南京理工大学博士学位论文,2002。
[8] YU-TE WU,TAKEO KANADE,CHING-CHUNG LI,JEFFREY COHN. Image Registration Using Wavelet-Based Motion Model. International Journal of Computer Vision 38(2), 129 -152 ,2000.
[9] Dimitrova N, Abdel-Mottaled M. Content-Based video retrieval by example video clip. In: Proceedings of IS&T and SPIE Storage and Retrieval of Image and Video Databases VI, Vol.3022. 1998. 184～196.
[10] Jain AK, Vailaya A, Wei X. Query by video clip. ACM Multimedia Systems, 1999, 7(5):369～384.
[11] Liu XM, Zhuang YT, Pan YH. A new approach to retrieve video by example video clip.In: Proceedings of ACM Multimedia.1999.41～44.
[12]李水根,吴纪桃,分形与小波,科学版,2002,pp196-316
[13]阮秋琦,数字图像处理学,电子工业版,2001, pp122-179
[14]陈武凡主编《.小波分析及其在图像处理中的应用》[M].北京:科学出出版社,2002。
[15]庄越挺,潘云鹤,吴飞编著.《网上多媒体信息分析与检索》[M].北京:清华大学出版社,2002.
[16] Wu Y, Zhuang YT, Pan YH. Content-Based video similarity model. In: Proceedings of the ACM Multimedia,2000.
[17] Zhuang YT, Liu XM, Wu Y, Pan YH. A new approach to retrieve video by example video clip. Chinese Journal of Computers, 2000,23(3):300～305.
[18] Chen LP, Chua TS. A match and tiling approach to content-based video retrieval. In: Proceedings of IEEE International Conference on Multimedia and Expo(ICME2001).2001.417～420.
[19] Orkun Alatas ,omar javed Mubarak Shah. Compressed Spatio-temporal Descriptors for video matching and retrieval.17th International Conference on Pattern Recognition (ICPR’04).Volume 3.
[20] Zhang H J, Wu Jianhua,. An Integrated System for Content-based Video Retrieval and Browsing[J ] . Pattern Recognition ,1997,Vol.30(4) page(s):643-657.
[21] Wolf Wayne. "Key frame selection by motion analysis". Proc. of IEEE Int. Conf. On Acoustics, Speech and Signal Processing, ICASSP, Atlanta,1996,page(s):7-10.
[22] Tekalp A Murat. Digital Video Processing[M] .北京:清华大学出版社,1998.
[23]彭宇新,Ngo Chong-Wah,董庆杰等.一种通过视频片段进行视频检索的方法.软件学报.2003.14(8).1409～1417
[24] Tong Lin ,et al. Video Content Representation for Shot Retrieval and Scene Extraction [J] . International Journal of Image and Graphics ,2001 ,1(3)
[25] Myung H,Senong K.A Multiplierless 2-D Convolver Chip for Real-Time Image Processing[J].Journal of VLSI Signal Processing,2004,38:page(s)63-71.
[26] Roy Utpal , Xu Yaoxian. Computation of a geometric model of a machined part from its NC machining programs[J ] . Computer-Aided Design ,1999 ,31 :401—411.
[27]宋传鸣,王相海.小波域视频运动估计研究进展.计算机学报.Vol.28 No.10,Oct.2005
[28] Cui Su-Xia. Motion estimation and compensation in the redundant wavelet domain [ Ph. D. dissertation ] . Mississippi State University , Mississippi State , USA , 2003
[29] J. Fridrich, M. Goljan, and D. Hogea, New methodology for breaking steganographic techniques for JPEGs,in SPIE Symposium on Electronic Imaging, (Santa Clara, CA), 2003
[30] Arredondo M A, lebart K, Lane D. Optical flow using textures[J]. Pattern Recognition Letters, 2004;25:449-457.
[31] Changbo Hu, Yi Li, Songde Ma et al. Region based parametric motion representation[J]. Pattern Recognition,2000,3:861-864.
[32] Vicente Grau, Mariano A R, Hierarchical image segmentation using a correspondence with a tree model[J]. Pattern Recognition, 2004;37:47-59.
[33]胡汉平,李德华,熊联欢等.基于小波变换的实时图像采集系统[J].数据采集与处理,1997,12(4):268-271.
[34]杨维明,周明,黄秋安. FFT专用芯片的应用[J].半导体技术,2002,27(2):54-56.
[35] Emanuel Radoi, Brigitte Hoeltzener. Improving the Radar Target Classification Results by Decision Fusion[C]. In: Proceedings of the International Radar Conference 2003,2003:162-165.
[36] Ganesan S A L. Texture classification using wavelet transform[J]. Pattern Recognition Letters, 2003,1(24):1513-1521.
[37] Ohm J R. A set of visual feature descriptors and their combination in a low-level description scheme[J]. Signal Processing: Image Communication,2000,9(16):157-179.
[38]黄卓君,马争鸣.多小波图像变换的统计分析[J].中国图像图形学报,2001,12(6):1198-2004.
[39] Tan YP, Kulkarni SR, Ramadge PJ. A framework for measuring video similarity and its application to video query by example .In: Proceedings of IEEE International Conference on Image Processing (ICIP 1999). 1999:106-110.
[40] Dimitrova N, Abdel-Mottaled M. Content-Based video retrieval by example video clip. In: Proceedings of IS&T and SPIE Storage and Retrieval of Image and Video Databases VI, Vol.3022.1998.184-196.
[41]林通,张宏江,封举富,石青云.镜头内容分析及其在视频检索中的应用.软件学报,2002,13(8):1577-1585.
[42]赵黎,祁卫,李子青,杨士强,张宏江.利用改进NFL算法对镜头进行基于内容的检索.软件学报,2002,13(4):586-590.
[43] Ekin A, Tekalp A M. Shot type classification by dominant color for sports video segmentation and summarization[A]. IEEE International Conference on Acoustics ,Speech, and Signal Processing 2003[C]. Hong Kong: IEEE Signal Processing Society, 2003.6-10.
[44] A Akharraz, Mauris G.A project Decision Support System Based on an Elucidative Fusion System[C]. In:Proceeding of the Fifth International Conference on Information Fusion 2002,2002-07:593-599.
[45] Jeannin S, Mory B. Video Motion Representation for Improved Content Access. IEEE Transactions on Consumer Electronics,2000,46(3).
[46] Ye Jiamin Smeaton A F. Aggregated Feature Retrieval for MPEG-7. ECIR 2003,LNCS 2633,2003:563-570.
[47] Jing H, Zhang H J, Video Segmentation with the Support of Audio Segmentation and Classification[J]. Proceedings of the IEEE Int. Conf. on Multimedia and Expo(ICME 2000),2000,3:1507-1510
[48]庄越挺.通过例子视频进行视频检索的新方法[J].计算机学报.2000,23(3):300-305.
[49] Meng J et al. Scene change detection in a MPEG compressed video sequence[C]. In:Proceedings of IS & T/SPIE International Symposium on Electronic Imaging, San Jose,1995:14-25.
[50] Frarg W, Abdel- Wahab H. A new paradigm for detecting scene changes on MPEG compressed videos[C], In: Proceedings of IEEE International Symposium on Signal Processing and Information Technology, Cairo, Egypt,2001:153-158.
[51] Change Sh2F, et al. VideoQ: An Automated Content-based Video Search System using Visual Cues[C]. Proc. Of ACM Multimedia’97.Seattle Washington USA, 1997.3132324
[52] Patke Nilesh V, Sethi lshwar K, Video shot detection and characterization for video databases, Pattern Recognition,vol 30,no 4,1997,583-592.