基于重叠区域相关系数的视频影像关键帧提取算法
|
摘要
提出了3种基于影像重叠区相关系数的视频影像关键帧快速提取算法。该算法首先采用多项式快速、准确地拟合出视频影像相关系数的变化趋势,然后利用相邻关键帧影像重叠区的高相关特性实现关键帧帧位的快速、准确提取,并通过实例与基于重叠度计算关键帧算法的时效性进行对比。结果表明,该算法单次计算相关系数的时间相对于单次重叠度计算时间缩短了近4倍,准确度达到88%,且针对不同拍摄手段下的视频影像,提取的关键帧影像精度较高。同时,还对基于视频提取的关键帧影像进行全景图拼接,结果显示未出现明显场景缺失,且可靠性高,适用性强。
This paper presents an effective algorithm to extract key frames from video images based on the correlation coefficients derived from the overlapping regions of adjacent images. Firstly, the polynomial model is employed to fit the change trend of correlation coefficients. Secondly, the positions of key frames are located and extracted rapidly and correctly via the high correlation of overlapping regions of adjacent key frame images. Finally, the time-effectiveness of extracting key frames based on the degree of overlap is tested in a stance. The result shows that the proposed algorithm takes only near 1/4 of time consumed by single computing of the degree of overlap, with an accuracy up to 88% and it can achieve higher accuracy for extracting key frames with different requires of degree of overlap between video images. The panchromatic image generated from the mosaic of extracted key video frames shows no apparent missing of scenes,and the new algorithm demonstrates high reliability and transferability.
This paper presents an effective algorithm to extract key frames from video images based on the correlation coefficients derived from the overlapping regions of adjacent images. Firstly, the polynomial model is employed to fit the change trend of correlation coefficients. Secondly, the positions of key frames are located and extracted rapidly and correctly via the high correlation of overlapping regions of adjacent key frame images. Finally, the time-effectiveness of extracting key frames based on the degree of overlap is tested in a stance. The result shows that the proposed algorithm takes only near 1/4 of time consumed by single computing of the degree of overlap, with an accuracy up to 88% and it can achieve higher accuracy for extracting key frames with different requires of degree of overlap between video images. The panchromatic image generated from the mosaic of extracted key video frames shows no apparent missing of scenes,and the new algorithm demonstrates high reliability and transferability.
引文
[1] Wolf W. Key Frame Selection by Motion Analysis[C]. IEEE International Conference on Acoustics, Speech & Signal Processing,Atlanta, Georgia, United States, 1996
[2] Zhang H J, Wu J, Zhong D, et al. An Integrated System for Content-based Video Retrieval and Browsing[J]. Pattern Recognition, 1997, 30(4):643-658
[3] Gresle P O,Huang T S.Gisting of Video Documents: A Key Frames Selection Algorithm Using Relative Activity Measure[C].The 2nd IntConf on Visual Information Systems,San Diego California, United States,1997
[4] Zhuang Y, Yong R, Huang T S, et al. Adaptive Key Frame Extraction Using Unsupervised Clustering[C].IEEE International Conference on Image Processing, Chicago, Illinois, USA, 1998
[5] Ferman A M, Tekalp A M. Multiscale Content Extraction and Representation for Video Indexing[J]. Proceedings of SPIE-The International Society for Optical Engineering, 1997, 3 229:23-31
[6] Lu Weiyan,Xia Dingyuan,Liu Yi.An Appooach of Key Frame Extraction Based on Mutual Information[J]. Microcomputer Information, 2007, 23(33):298-300(陆伟艳, 夏定元, 刘毅. 基于内容的视频检索的关键帧提取[J]. 微计算机信息, 2007, 23(33):298-300)
[7] Zhu Dengming,Wang Zhaoqi. Extraction of Keyframe from Motion Capture Data Based on Motion Sequence Segmentation[J]. Journal of Computer-Aided Design & Computer Graphics,2008, 20(6):787-792(朱登明, 王兆其. 基于运动序列分割的运动捕获数据关键帧提取[J]. 计算机辅助设计与图形学学报, 2008, 20(6):787-792 )
[8] Li J, Pan Q, Yang T, et al. Automated Feature Points Management for Video Mosaic Construction[C].IEEE International Conference on Information Technology & Applications,Sydney, Australia, 2005
[9] Szeliski R. Video Mosaics for Virtual Environments[J]. IEEE Computer Graphics & Applications, 1996, 16(2):22-30
[10] De Haan G, Biezen P W A C, Huijgen H, et al. True-motion Estimation with 3-D Recursive Search Block Matching[J]. IEEE Transactions on Circuits & Systems for Video Technology, 1993, 3(5):368-379, 388
[11] Zhang Jianqing, Pan Li.Photogrammetry[M].2nd ed. Wuhan:Wuhan University Press, 2012(张剑清, 潘励. 摄影测量学)[M].2版.武汉:武汉大学出版社,2012)
[12] Liu Yong,Wang Guijin, Yao Anbang, et al.Video Mosaicking Based on Adaptive Sampling[J].J Tsinghua Univ(Sci & Tech) ,2010,50(1):108-112(刘永, 王贵锦, 姚安邦,等. 基于自适应帧采样的视频拼接[J]. 清华大学学报(自然科学版), 2010,50(1):108-112)
[13] Fadaeieslam M J, Soryani M, Fathy M. Efficient Key Frames Selection for Panorama Generation from Video[J]. Journal of Electronic Imaging, 2011, 20(2):2 763-2 769
[14] Lin Zhongjian,Zhang Yonghong.Measurement of Information and Uncertainty of Remote Sensing and GIS Data[J].Geomatics and Information Science of Wnhan University,2006, 31(7):569-572(林宗坚,张永红.遥感与地理信息系统数据的信息量及不确定性[J].武汉大学学报·信息科学版,2006, 31(7):569-572)
[15] Li Chaokui,Zhao Huibing,Yang Gang. Research on Separating Technology of UAVs Video Image Stream Based on the POS System[J].Journal of Hunan University of Science & Technology(Natural Science Edition),2009, 24(3):51-55(李朝奎, 赵会兵, 杨刚. 基于POS系统的无人机视频影像分割技术研究[J]. 湖南科技大学学报(自然科学版), 2009, 24(3):51-55)
[16] Liu Shanlei,Zhao Yingdi,Wang Ganghui,et al. A Method for Automatic Extraction of Key Frames[J].Science of Surveying and Mapping,2012, 37(5):110-115(刘善磊, 赵银娣, 王光辉,等. 一种关键帧的自动提取方法[J]. 测绘科学,2012, 37(5):110-115)
[17] Xing Cheng.Overlapping Degree Analysis of Images from an Unmanned Aerial Yehicle Based on a Reduced Scale-Invariant Feature Transform (SIFT) Algorithm[J].Journal of Harbin Engineering University,2012, 33(2):221-225(邢诚. 基于简化SIFT算法的无人机影像重叠度分析[J]. 哈尔滨工程大学学报, 2012, 33(2):221-225)
[18] Liu Chang’an,Sun Jin,Wu Hua,et al.Research on Keyframe Extraction Pretreatment of Power-tower in Flying Robot Inspection Video[J]. J Huazhong Univ of Sci & Tech(Natural Science Edition) 2015,43(s1):477-480(柳长安,孙金,吴华,等.飞行机器人电塔巡检视频关键帧提取预处理[J].华中科技大学学报(自然科学版), 2015,43(s1):477-480 )
[2] Zhang H J, Wu J, Zhong D, et al. An Integrated System for Content-based Video Retrieval and Browsing[J]. Pattern Recognition, 1997, 30(4):643-658
[3] Gresle P O,Huang T S.Gisting of Video Documents: A Key Frames Selection Algorithm Using Relative Activity Measure[C].The 2nd IntConf on Visual Information Systems,San Diego California, United States,1997
[4] Zhuang Y, Yong R, Huang T S, et al. Adaptive Key Frame Extraction Using Unsupervised Clustering[C].IEEE International Conference on Image Processing, Chicago, Illinois, USA, 1998
[5] Ferman A M, Tekalp A M. Multiscale Content Extraction and Representation for Video Indexing[J]. Proceedings of SPIE-The International Society for Optical Engineering, 1997, 3 229:23-31
[6] Lu Weiyan,Xia Dingyuan,Liu Yi.An Appooach of Key Frame Extraction Based on Mutual Information[J]. Microcomputer Information, 2007, 23(33):298-300(陆伟艳, 夏定元, 刘毅. 基于内容的视频检索的关键帧提取[J]. 微计算机信息, 2007, 23(33):298-300)
[7] Zhu Dengming,Wang Zhaoqi. Extraction of Keyframe from Motion Capture Data Based on Motion Sequence Segmentation[J]. Journal of Computer-Aided Design & Computer Graphics,2008, 20(6):787-792(朱登明, 王兆其. 基于运动序列分割的运动捕获数据关键帧提取[J]. 计算机辅助设计与图形学学报, 2008, 20(6):787-792 )
[8] Li J, Pan Q, Yang T, et al. Automated Feature Points Management for Video Mosaic Construction[C].IEEE International Conference on Information Technology & Applications,Sydney, Australia, 2005
[9] Szeliski R. Video Mosaics for Virtual Environments[J]. IEEE Computer Graphics & Applications, 1996, 16(2):22-30
[10] De Haan G, Biezen P W A C, Huijgen H, et al. True-motion Estimation with 3-D Recursive Search Block Matching[J]. IEEE Transactions on Circuits & Systems for Video Technology, 1993, 3(5):368-379, 388
[11] Zhang Jianqing, Pan Li.Photogrammetry[M].2nd ed. Wuhan:Wuhan University Press, 2012(张剑清, 潘励. 摄影测量学)[M].2版.武汉:武汉大学出版社,2012)
[12] Liu Yong,Wang Guijin, Yao Anbang, et al.Video Mosaicking Based on Adaptive Sampling[J].J Tsinghua Univ(Sci & Tech) ,2010,50(1):108-112(刘永, 王贵锦, 姚安邦,等. 基于自适应帧采样的视频拼接[J]. 清华大学学报(自然科学版), 2010,50(1):108-112)
[13] Fadaeieslam M J, Soryani M, Fathy M. Efficient Key Frames Selection for Panorama Generation from Video[J]. Journal of Electronic Imaging, 2011, 20(2):2 763-2 769
[14] Lin Zhongjian,Zhang Yonghong.Measurement of Information and Uncertainty of Remote Sensing and GIS Data[J].Geomatics and Information Science of Wnhan University,2006, 31(7):569-572(林宗坚,张永红.遥感与地理信息系统数据的信息量及不确定性[J].武汉大学学报·信息科学版,2006, 31(7):569-572)
[15] Li Chaokui,Zhao Huibing,Yang Gang. Research on Separating Technology of UAVs Video Image Stream Based on the POS System[J].Journal of Hunan University of Science & Technology(Natural Science Edition),2009, 24(3):51-55(李朝奎, 赵会兵, 杨刚. 基于POS系统的无人机视频影像分割技术研究[J]. 湖南科技大学学报(自然科学版), 2009, 24(3):51-55)
[16] Liu Shanlei,Zhao Yingdi,Wang Ganghui,et al. A Method for Automatic Extraction of Key Frames[J].Science of Surveying and Mapping,2012, 37(5):110-115(刘善磊, 赵银娣, 王光辉,等. 一种关键帧的自动提取方法[J]. 测绘科学,2012, 37(5):110-115)
[17] Xing Cheng.Overlapping Degree Analysis of Images from an Unmanned Aerial Yehicle Based on a Reduced Scale-Invariant Feature Transform (SIFT) Algorithm[J].Journal of Harbin Engineering University,2012, 33(2):221-225(邢诚. 基于简化SIFT算法的无人机影像重叠度分析[J]. 哈尔滨工程大学学报, 2012, 33(2):221-225)
[18] Liu Chang’an,Sun Jin,Wu Hua,et al.Research on Keyframe Extraction Pretreatment of Power-tower in Flying Robot Inspection Video[J]. J Huazhong Univ of Sci & Tech(Natural Science Edition) 2015,43(s1):477-480(柳长安,孙金,吴华,等.飞行机器人电塔巡检视频关键帧提取预处理[J].华中科技大学学报(自然科学版), 2015,43(s1):477-480 )