基于边缘特征和自适应融合的视频显著性检测
|
摘要
针对目前大多数视频显著性检测中背景复杂以及显著目标边缘模糊、显著目标内部存在空洞不能一致高亮的问题,提出了一种基于动静态边缘和自适应融合的视频显著性检测算法。该算法利用静态边缘和运动边缘信息融合后初步定位显著目标,并对其进行一系列平滑操作获得目标的精确边缘然后计算梯度获得初始显著图。然后,考虑前一帧对当前帧的有效性约束,计算相邻两帧的颜色直方图进而得到两帧的相似度,由相似度决定两帧在自适应融合时各自的比重,得到当前帧的最终显著图。在公开视频显著性数据集ViSal上算法F值接近0.8,MAE接近0.06,表明该方法性能优于目前主流算法,对复杂背景有较强鲁棒性,同时能够快速、清晰而准确地提取出视频序列中的显著性目标。
The present video saliency detection methods can′t extract salient objects in complex backgrounds efficiently and the salient objects cannot be uniformly highlighted. A video saliency detection algorithm based on edge extraction and adaptive fusion is proposed. The algorithm first uses the static edge and motion edge to locate the salient object, and then calculates the gradient to obtain the initial saliency map. Secondly, considering the validity constraint of the previous frame on the current frame, the color histogram of the two adjacent frames is used to calculate the similarity of the two frames, which can determine the respective weights of the two frames when fused to calculate the final salient maps.The experiments show the F-score of the proposed algorithm is close to 0.8, and the MAE is close to 0.06. The proposed method is superior to the state-of-the-art algorithms, robust to complex background.
The present video saliency detection methods can′t extract salient objects in complex backgrounds efficiently and the salient objects cannot be uniformly highlighted. A video saliency detection algorithm based on edge extraction and adaptive fusion is proposed. The algorithm first uses the static edge and motion edge to locate the salient object, and then calculates the gradient to obtain the initial saliency map. Secondly, considering the validity constraint of the previous frame on the current frame, the color histogram of the two adjacent frames is used to calculate the similarity of the two frames, which can determine the respective weights of the two frames when fused to calculate the final salient maps.The experiments show the F-score of the proposed algorithm is close to 0.8, and the MAE is close to 0.06. The proposed method is superior to the state-of-the-art algorithms, robust to complex background.
引文
[1] ROTHER C,KOLMOGOROV V,BLAKE A. Grabcut:interactive foreground extraction using iterated graph cuts[J]. ACM Transactions on Graphics,2004,23(1):309-314.
[2] MAHADEVAN V,VASOONCEIOS N. Saliency-based discriminant tracking[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR),Miami,USA,2009:1007-1013.
[3]肖德贵,辛晨,张婷,等.显著性纹理结构特征及车载环境下的行人检测[J].软件学报,2014,25(3):675-689.
[4] ITTI L,KOCH C,and NIEBUR E. A model of saliency-based visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[5] CHENG M M,ZHANG G X,MITRA N J,et al. Global contrast based salient region detection[C]//IEEE International Conference on Computer Vision and Pattern Recognition,Colorado Springs,2011:409-416.
[6] ZHU W J,LIANG S,WEI Y C,et al. Saliency Optimization from Robust Background Detection[C]//IEEE International Conference on Computer Vi?sion and Pattern Recognition,Columbus,2014:2814-2821.
[7]蒋寓文,谭乐怡,王守觉.选择性背景优先的显著性检测模型[J].电子与信息学报,2015,37(1):130-136.
[8] GUO C L,MA Q,ZHANG L M. Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform[C]//Computer Vision and Pattern Recognition,Anchorage,AK,USA,2007:1-8.
[9] HOU X D,ZHANG L Q. Saliency Detection:A spectral residual approach[C]//IEEE Conference on Computer Vision and Pattern Recognition,Minne?apolis,Minnesota,USA,2007:1-8.
[10]田畅,姜青竹,吴泽民,等.基于区域协方差的视频显著度局部空时优化模型[J].电子与信息学报,2016,38(7):1586-1593.
[11] KIM W,KIM C. Spatiotemporal saliency detection using textural contrast and its applications[J]. IEEE Transactions on Circuits&Systems for Video Technology,2014,24(4):646-659.
[12] ZHOU F,KANG S B,COHEN M F. Time-mapping using space-time saliency[C]//IEEE Conference on Computer Vision and Pattern Recognition,Columbus,USA,2014:3358-3365.
[13] DESIMONE R,DUNCAN J. Neural mechanisms of selective visual attention[J]. Annual review of neuroscience,1995,18(1):193-222.
[14] ACHANTA R,SHAJI A,SMITH K,et al. SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Anal?ysis and Machine Intelligence,2012,34(11):2274-2281.
[15] BROX T,MALIK J. Large displacement optical flow:descriptor matching in variational motion estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,33(3):500-513.
[16] WANG W G,SHEN J B,and SHAO L. Consistent Video Saliency Using Local Gradient Flow Optimization and Global Refinement[J]. IEEE Transac?tions on Image Processing,2015,24(10):1-12.
[17] WANG W G,SHEN J B,PORIKLI F. Saliency-aware geodesic video object segmentation[C]//IEEE Conference on Computer Vision and Pattern Rec?ognition,Boston,USA,2015:3395-3402.
[2] MAHADEVAN V,VASOONCEIOS N. Saliency-based discriminant tracking[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR),Miami,USA,2009:1007-1013.
[3]肖德贵,辛晨,张婷,等.显著性纹理结构特征及车载环境下的行人检测[J].软件学报,2014,25(3):675-689.
[4] ITTI L,KOCH C,and NIEBUR E. A model of saliency-based visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[5] CHENG M M,ZHANG G X,MITRA N J,et al. Global contrast based salient region detection[C]//IEEE International Conference on Computer Vision and Pattern Recognition,Colorado Springs,2011:409-416.
[6] ZHU W J,LIANG S,WEI Y C,et al. Saliency Optimization from Robust Background Detection[C]//IEEE International Conference on Computer Vi?sion and Pattern Recognition,Columbus,2014:2814-2821.
[7]蒋寓文,谭乐怡,王守觉.选择性背景优先的显著性检测模型[J].电子与信息学报,2015,37(1):130-136.
[8] GUO C L,MA Q,ZHANG L M. Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform[C]//Computer Vision and Pattern Recognition,Anchorage,AK,USA,2007:1-8.
[9] HOU X D,ZHANG L Q. Saliency Detection:A spectral residual approach[C]//IEEE Conference on Computer Vision and Pattern Recognition,Minne?apolis,Minnesota,USA,2007:1-8.
[10]田畅,姜青竹,吴泽民,等.基于区域协方差的视频显著度局部空时优化模型[J].电子与信息学报,2016,38(7):1586-1593.
[11] KIM W,KIM C. Spatiotemporal saliency detection using textural contrast and its applications[J]. IEEE Transactions on Circuits&Systems for Video Technology,2014,24(4):646-659.
[12] ZHOU F,KANG S B,COHEN M F. Time-mapping using space-time saliency[C]//IEEE Conference on Computer Vision and Pattern Recognition,Columbus,USA,2014:3358-3365.
[13] DESIMONE R,DUNCAN J. Neural mechanisms of selective visual attention[J]. Annual review of neuroscience,1995,18(1):193-222.
[14] ACHANTA R,SHAJI A,SMITH K,et al. SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Anal?ysis and Machine Intelligence,2012,34(11):2274-2281.
[15] BROX T,MALIK J. Large displacement optical flow:descriptor matching in variational motion estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,33(3):500-513.
[16] WANG W G,SHEN J B,and SHAO L. Consistent Video Saliency Using Local Gradient Flow Optimization and Global Refinement[J]. IEEE Transac?tions on Image Processing,2015,24(10):1-12.
[17] WANG W G,SHEN J B,PORIKLI F. Saliency-aware geodesic video object segmentation[C]//IEEE Conference on Computer Vision and Pattern Rec?ognition,Boston,USA,2015:3395-3402.