基于HEVC压缩域的镜头边界检测方法
|
摘要
镜头边界检测是智能视频检索的一个重要环节。现有的检测方法主要是在像素域进行处理,切变检测精度不高,计算复杂度过大。针对这些问题,文中利用解析HEVC码流得到的编码信息,提出了一种基于HEVC压缩域的镜头边界检测方法。首先统计每帧编码信息中各类预测模式的PU个数,并根据CU深度对运动矢量进行幅值滤波;然后采用PU预测模式、运动矢量和帧比特数对切变候选帧进行两级筛选,再对其进行自适应阈值的镜头切变检测;接着根据切变帧对视频序列进行分段,并在时域上对帧比特数进行平滑滤波;最后使用PU预测模式和经滤波平滑后的帧比特数对分段视频进行镜头渐变检测。实验结果表明,该方法具有良好的镜头边界检测效果,并具有较低的计算复杂度。
Shot boundary detection is an important part of intelligent video retrieval.The existing detection methods are mainly processed in pixel domain,with low accuracy of cut and high computational complexity.To solve these problems,this paper used the encoding information obtained by parsing HEVC stream and proposesd a shot boundary detection method based on HEVC compressed domain.First,the number of PUs with different prediction modes is counted for each frame,and the motion vectors are filtered according to CU depth.Then,the two-stage candidate frame of cut is selected by using the PU prediction modes,the motion vectors and the number of frame bits.And then,the cut shot detection of adaptive threshold is performed.After that,the video is segmented according to the cut frame.In addition,the smooth filtering is carried out for the frame bits in the time domain.Finally,the PU prediction modes and the number of smoothed frame bits are used to detect the gradual shot detection.The experimental results show that the proposed method has a good effect on shot boundary detection with lower computational complexity.
Shot boundary detection is an important part of intelligent video retrieval.The existing detection methods are mainly processed in pixel domain,with low accuracy of cut and high computational complexity.To solve these problems,this paper used the encoding information obtained by parsing HEVC stream and proposesd a shot boundary detection method based on HEVC compressed domain.First,the number of PUs with different prediction modes is counted for each frame,and the motion vectors are filtered according to CU depth.Then,the two-stage candidate frame of cut is selected by using the PU prediction modes,the motion vectors and the number of frame bits.And then,the cut shot detection of adaptive threshold is performed.After that,the video is segmented according to the cut frame.In addition,the smooth filtering is carried out for the frame bits in the time domain.Finally,the PU prediction modes and the number of smoothed frame bits are used to detect the gradual shot detection.The experimental results show that the proposed method has a good effect on shot boundary detection with lower computational complexity.
引文
[1] DAI K X,LI Q,LI G H.Prospects and current studies on video mining[J].Computer Science,2010,37(10):11-15.(in Chinese)代科学,李强,李国辉.视频挖掘研究进展[J].计算机科学,2010,37(10):11-15.
[2] SHEN R K,LIN Y N,JUANG T T,et al.Automatic detection of video shot boundary in social media using a hybrid approach of HLFPN and keypoint matching[J].IEEE Transactions on Computational Social Systems,2018,5(1):210-219.
[3] ALI M,ADNAN A.Short boundary detection using spatial-temporal features[J].Advances in Intelligent Systems and Computing,2016,448:971-981.
[4] SANTOS A C S,PEDRINI H.Shot boundary detection for video temporal segmentation based on the weber local descriptor[C]∥IEEE International Conference on Systems,Man,and Cybernetics.IEEE,2017:1310-1315.
[5] LI J Q,PAN Q K,LIANG Y C.Shot change detection on news videos using color histogram and edge based approaches[C]∥IEEE International Conference on Advances in Computer Applications.IEEE,2016:50-54.
[6] PENG T L,ZHANG W J,WANG Y B,et al.Video shot boundary detection algorithm based on multi-features[J].Chinese Journal of Scientific Instrument,2015,36(9):2013-2020.(in Chinese)彭太乐,张文俊,汪友宝,等.基于多特征的视频镜头检测方法[J].仪器仪表学报,2015,36(9):2013-2020.
[7] QU Z,GAO T F,ZHANG Q Q.Study on an improved algorithm of video keyframe extraction[J].Computer Science,2012,39(8):300-303.(in Chinese)瞿中,高腾飞,张庆庆.一种改进的视频关键帧提取算法研究[J].计算机科学,2012,39(8):300-303.
[8] ZHONG X,YANG G,LU Y S.Method of key frames extraction based on double-threshold values sliding window sub-shot segmentation and fully connected graph[J].Computer Science,2016,43(6):289-293.(in Chinese)钟忺,杨光,卢炎生.基于双阈值滑动窗口子镜头分割和完全连通图的关键帧提取方法[J].计算机科学,2016,43(6):289-293.
[9] LU Z M,SHI Y.Fast video shot boundary detection based on SVD and pattern matching[J].IEEE Transactions on Image Processing,2013,22(12):5136-5145.
[10]LAKSHMI P G G,DOMNIC S.Walsh-hadamard transform kernel-based feature vector for shot boundary detection[J].IEEE Transactions on Image Processing,2014,23(12):5187-5197.
[11]TIPPAYA S,SITJONGSATAPORN S,TAN T,et al.MultiModal Visual Features-Based Video Shot Boundary Detection[J].IEEE Access,2017,5:12563-12575.
[12]MONDAL J,KUNDU M K,DAS S,et al.Video shot boundary detection using multiscale geometric analysis of nsct and least squares support vector machine[J].Multimedia Tools and Applications,2018,77(7):8139-8161.
[13]JIAN M,YIN Y,DONG J.Relative Flow Estimates for Shot Boundary Detection[J].Pattern Recognition and Image Analysis,2018,28(1):53-58.
[14]YANG Z,LI C.Gradual shot detection employing automatic white balance method[C]∥ACM International Conference on Multimedia Systems and Signal Processing.ACM,2018:71-74.
[15]SULLIVAN G J,OHM J,HAN W J,et al.Overview of the high efficiency video coding(HEVC)standard[J].IEEE Transactions on Circuits&Systems for Video Technology,2012,22(12):1649-1668.
[16]GONG S R,FAN Y J,ZHOU X.A novel scene change detection algorithm on H.264/AVC[J].Journal of Chinese Computer Systems,2007,28(4):688-691.(in Chinese)龚声蓉,范益进,周翔.一种基于H.264/AVC码流的镜头边界检测方法[J].小型微型计算机系统,2007,28(4):688-691.
[17]XIA D Y,XIE H L.Shot boundary detection based on H.264/AVC compressed domain[J].Journal of Image and Graphics,2009,14(12):2595-2598.(in Chinese)夏定元,谢惠琳.一种在H.264/AVC压缩域中检测镜头边界的方法[J].中国图象图形学报,2009,14(12):2595-2598.
[18]LIU Y,WANG W,GAO W,et al.A novel compressed domain shot segmentation algorithm on H.264/AVC[C]∥IEEE International Conference on Image Processing.IEEE,2004:2235-2238.
[19]ZHANG W,WANG Y,JIANG X.A shot segmentation algorithm for H.264compressed videos[C]∥IEEE International Congress on Image and Signal Processing.IEEE,2013:81-85.
[20]ZHANG W,WANG Y,JIANG X.A compressed-domain method of shot segmentation for X264videos[C]∥IEEE International Conference on Natural Computation.IEEE,2014:868-872.
[21]YOU Y X,ZHANG E D,GOU Z J.Shot boundary detection using Biased-SVM in H.264compressed domain[J].Computer Engineering and Applications,2013,49(24):138-143.(in Chinese)游运喜,张恩迪,苟志坚.H.264压缩域中利用Biased-SVM检测镜头边界[J].计算机工程与应用,2013,49(24):138-143.
[22]ZHANG Q M.Video shot boundary detection based on MB coding mode and SIFT features on H.264/AVC[C]∥IEEE International Conference on Progress in Informatics and Computing.IEEE,2014:299-302.
[2] SHEN R K,LIN Y N,JUANG T T,et al.Automatic detection of video shot boundary in social media using a hybrid approach of HLFPN and keypoint matching[J].IEEE Transactions on Computational Social Systems,2018,5(1):210-219.
[3] ALI M,ADNAN A.Short boundary detection using spatial-temporal features[J].Advances in Intelligent Systems and Computing,2016,448:971-981.
[4] SANTOS A C S,PEDRINI H.Shot boundary detection for video temporal segmentation based on the weber local descriptor[C]∥IEEE International Conference on Systems,Man,and Cybernetics.IEEE,2017:1310-1315.
[5] LI J Q,PAN Q K,LIANG Y C.Shot change detection on news videos using color histogram and edge based approaches[C]∥IEEE International Conference on Advances in Computer Applications.IEEE,2016:50-54.
[6] PENG T L,ZHANG W J,WANG Y B,et al.Video shot boundary detection algorithm based on multi-features[J].Chinese Journal of Scientific Instrument,2015,36(9):2013-2020.(in Chinese)彭太乐,张文俊,汪友宝,等.基于多特征的视频镜头检测方法[J].仪器仪表学报,2015,36(9):2013-2020.
[7] QU Z,GAO T F,ZHANG Q Q.Study on an improved algorithm of video keyframe extraction[J].Computer Science,2012,39(8):300-303.(in Chinese)瞿中,高腾飞,张庆庆.一种改进的视频关键帧提取算法研究[J].计算机科学,2012,39(8):300-303.
[8] ZHONG X,YANG G,LU Y S.Method of key frames extraction based on double-threshold values sliding window sub-shot segmentation and fully connected graph[J].Computer Science,2016,43(6):289-293.(in Chinese)钟忺,杨光,卢炎生.基于双阈值滑动窗口子镜头分割和完全连通图的关键帧提取方法[J].计算机科学,2016,43(6):289-293.
[9] LU Z M,SHI Y.Fast video shot boundary detection based on SVD and pattern matching[J].IEEE Transactions on Image Processing,2013,22(12):5136-5145.
[10]LAKSHMI P G G,DOMNIC S.Walsh-hadamard transform kernel-based feature vector for shot boundary detection[J].IEEE Transactions on Image Processing,2014,23(12):5187-5197.
[11]TIPPAYA S,SITJONGSATAPORN S,TAN T,et al.MultiModal Visual Features-Based Video Shot Boundary Detection[J].IEEE Access,2017,5:12563-12575.
[12]MONDAL J,KUNDU M K,DAS S,et al.Video shot boundary detection using multiscale geometric analysis of nsct and least squares support vector machine[J].Multimedia Tools and Applications,2018,77(7):8139-8161.
[13]JIAN M,YIN Y,DONG J.Relative Flow Estimates for Shot Boundary Detection[J].Pattern Recognition and Image Analysis,2018,28(1):53-58.
[14]YANG Z,LI C.Gradual shot detection employing automatic white balance method[C]∥ACM International Conference on Multimedia Systems and Signal Processing.ACM,2018:71-74.
[15]SULLIVAN G J,OHM J,HAN W J,et al.Overview of the high efficiency video coding(HEVC)standard[J].IEEE Transactions on Circuits&Systems for Video Technology,2012,22(12):1649-1668.
[16]GONG S R,FAN Y J,ZHOU X.A novel scene change detection algorithm on H.264/AVC[J].Journal of Chinese Computer Systems,2007,28(4):688-691.(in Chinese)龚声蓉,范益进,周翔.一种基于H.264/AVC码流的镜头边界检测方法[J].小型微型计算机系统,2007,28(4):688-691.
[17]XIA D Y,XIE H L.Shot boundary detection based on H.264/AVC compressed domain[J].Journal of Image and Graphics,2009,14(12):2595-2598.(in Chinese)夏定元,谢惠琳.一种在H.264/AVC压缩域中检测镜头边界的方法[J].中国图象图形学报,2009,14(12):2595-2598.
[18]LIU Y,WANG W,GAO W,et al.A novel compressed domain shot segmentation algorithm on H.264/AVC[C]∥IEEE International Conference on Image Processing.IEEE,2004:2235-2238.
[19]ZHANG W,WANG Y,JIANG X.A shot segmentation algorithm for H.264compressed videos[C]∥IEEE International Congress on Image and Signal Processing.IEEE,2013:81-85.
[20]ZHANG W,WANG Y,JIANG X.A compressed-domain method of shot segmentation for X264videos[C]∥IEEE International Conference on Natural Computation.IEEE,2014:868-872.
[21]YOU Y X,ZHANG E D,GOU Z J.Shot boundary detection using Biased-SVM in H.264compressed domain[J].Computer Engineering and Applications,2013,49(24):138-143.(in Chinese)游运喜,张恩迪,苟志坚.H.264压缩域中利用Biased-SVM检测镜头边界[J].计算机工程与应用,2013,49(24):138-143.
[22]ZHANG Q M.Video shot boundary detection based on MB coding mode and SIFT features on H.264/AVC[C]∥IEEE International Conference on Progress in Informatics and Computing.IEEE,2014:299-302.