复杂场景下的交通视频显著性前景目标提取
|
摘要
目的在城市交通检测中,智能视频的广泛应用使得人工智能技术及计算机视觉先进技术对视频中的前景目标检索、识别、特征提取、行为分析等成为视觉研究的热点,但由于复杂场景中动态背景具有不连续的特点,使得少部分的前景目标图像信息丢失,从而造成漏检、误判。方法本文提出一种RPCA(鲁棒主成分分析)优化方法,为了快速筛选与追踪前景目标,以基于帧差欧氏距离方法设计显著性目标帧号快速提取算法,确定关键帧邻域内为检测范围,对经过稀疏低秩模型初筛选的前景目标图像进行前景目标种子并行识别和优化连接,去除前景目标图像中的动态背景,同时将MASK(掩膜)图像中的前景目标分为规则类和非规则类两种,对非规则类前景目标如行人、动物等出现的断层分离现象设计前景目标区域纵向种子生长算法,对规则类前景目标如汽车轮船等设计区域内前景目标种子横纵双向连接以消除空洞、缺失的影响。结果本文前景目标提取在富有挑战性干扰因素的复杂场景下体现出较高的鲁棒性,在数据库4组经典视频和山西太长高速公路2组视频中,动态背景有水流流动、树叶摇曳、摄像头轻微抖动、光照阴影,并从应用效果、前景目标定位的准确性以及前景目标检测的完整性3个角度对实验结果进行了分析,本文显著性前景目标提取算法取得了90. 1%的平均准确率,88. 7%的平均召回率以及89. 4%的平均F值,均优于其他同类算法。结论本文以快速定位显著性前景目标为前提,提出对稀疏低秩模型初筛选的图像进行并行种子识别和优化连接算法,实验数据的定性与定量分析结果表明,本文算法能够更快速地将前景目标与动态背景分离,并减小前景目标与背景之间的粘连情况,更有效地保留了原始图像中前景目标的结构信息。
Objective In urban traffic detection,the wide application of intelligent video surveillance provides the visual research interest on artificial intelligence and advanced computer vision technology to retrieve and recognize the foreground object in video and its further analysis,such as feature extraction and abnormal behavior analysis. However,when facing complex environment,the discontinuity of the dynamic background causes loss of a small part of the future target image information,false detection,and misjudgment. Constructing an effective and high-performance extractor has two core issues.The first issue is the detection of speed and efficiency. If the video object can be extracted in advance and can determine which video frames do not contain the foreground object,it is directly eliminated in the earlier period,only concerning the image with a significant foreground target,which greatly improves the detection efficiency,because of the large video data.The second problem involves the object integrity in complex environments. Effectively extracting the foreground part of the video sequence becomes the key to the reliability of subsequent algorithms. Method This paper proposes a robust principal component analysis( RPCA) optimization method. The classical RPCA detection method uses the l0-norm to independently determine whether each pixel is a moving target and is not conducive to eliminate unstructured sparse components due to noise and random background disturbance. This paper aims to maintain the good robustness of the algorithm in the complex environment and optimize the RPCA initial filtered image. In order to quickly screen and track the foreground target,a fast extraction algorithm for the saliency target frame number is designed based on the frame difference Euclidean distance method to determine the detection range in the key frame neighborhood. Through the establishment and the solution of sparse low-rank models and based on the initially filtered foreground target image,parallel recognition of the foreground target seed is performed to remove the dynamic background in the foreground target image. Also,as observed from several mask images after gray value inversion,the foreground target pixel has a small gray value and strong directionality. Therefore,the design ideas for the parallel recognition and optimization connection method of the foreground target seed are: 1) By using gray pixel seed recognition,gray value inversion of the source image,and verification according to the gray scale and symmetry detection,grayscale pixels are identified as foreground and non-foreground target sub-blocks; 2) Grayscale pixels are optimized for connection,and foreground target seeds are connected according to grayscale values and directional similarity,followed by fusion and multi-template denoising; 3) Seed filling is used for foreground targets to enhance the connectivity and make the target more complete. Simultaneously,the foreground objects in the mask image are classified into regular and irregular class. For the fault separation of irregular targets such as pedestrians and animals,the vertical seed growth algorithm is designed in the target region. For the foreground targets of rules such as car and ships,the foreground seed in the design region is vertically and horizontally connected to remove the holes and the impact of the lack of structural information. Result The foreground target extraction is highly robust in complex environment with challenging interference factors.In the four groups of classic video of the database and the two videos of Shanxi Taichang Expressway,the dynamic background has the flow of water,swaying leaves,the slight jitter of the camera,and the change of light shadows. In addition,the experimental results were analyzed from three perspectives of the application effect,the accuracy of foreground target location,and the integrity of foreground target detection. The significance of target extraction algorithm has achieved an average accuracy of 90. 1%,an average recall of 88. 7%,and an average F value of 89. 4%,which are all superior to other similar algorithms. Compared with the mixed Gaussian model and the optical flow algorithm,the complex background brings a large noise disturbance. The Gaussian mixture model uses a morphological algorithm to remove the noise filling holes,giving the detected foreground target more viscous information. At different shadow area,the detection effect varies greatly.Furthermore,the optical flow algorithm is sensitive to light,and the changed light is mistaken for optical flow,which is not suitable under strict environmental requirements. Conclusion In this paper,by quickly locating the salient foreground,a parallel seed identification and optimized connection algorithm for RPCA initial screening image is proposed. The qualitative and quantitative analyses of the experimental data show that the algorithm can separate the foreground target from the dynamic background more quickly,reduce the adhesion between the foreground object and the background,and more effectively retain the structural information of the foreground object in the original image. In the following studies,deficiencies in the overall model and the algorithm details are continuously optimized. In the face of abnormal light rays,shadow suppression can be combined to make it more robust,and the performance and effectiveness of the algorithm are improved in more complex environments such as drone mobile video,which provides data support for feature extraction and abnormal behavior analysis.
Objective In urban traffic detection,the wide application of intelligent video surveillance provides the visual research interest on artificial intelligence and advanced computer vision technology to retrieve and recognize the foreground object in video and its further analysis,such as feature extraction and abnormal behavior analysis. However,when facing complex environment,the discontinuity of the dynamic background causes loss of a small part of the future target image information,false detection,and misjudgment. Constructing an effective and high-performance extractor has two core issues.The first issue is the detection of speed and efficiency. If the video object can be extracted in advance and can determine which video frames do not contain the foreground object,it is directly eliminated in the earlier period,only concerning the image with a significant foreground target,which greatly improves the detection efficiency,because of the large video data.The second problem involves the object integrity in complex environments. Effectively extracting the foreground part of the video sequence becomes the key to the reliability of subsequent algorithms. Method This paper proposes a robust principal component analysis( RPCA) optimization method. The classical RPCA detection method uses the l0-norm to independently determine whether each pixel is a moving target and is not conducive to eliminate unstructured sparse components due to noise and random background disturbance. This paper aims to maintain the good robustness of the algorithm in the complex environment and optimize the RPCA initial filtered image. In order to quickly screen and track the foreground target,a fast extraction algorithm for the saliency target frame number is designed based on the frame difference Euclidean distance method to determine the detection range in the key frame neighborhood. Through the establishment and the solution of sparse low-rank models and based on the initially filtered foreground target image,parallel recognition of the foreground target seed is performed to remove the dynamic background in the foreground target image. Also,as observed from several mask images after gray value inversion,the foreground target pixel has a small gray value and strong directionality. Therefore,the design ideas for the parallel recognition and optimization connection method of the foreground target seed are: 1) By using gray pixel seed recognition,gray value inversion of the source image,and verification according to the gray scale and symmetry detection,grayscale pixels are identified as foreground and non-foreground target sub-blocks; 2) Grayscale pixels are optimized for connection,and foreground target seeds are connected according to grayscale values and directional similarity,followed by fusion and multi-template denoising; 3) Seed filling is used for foreground targets to enhance the connectivity and make the target more complete. Simultaneously,the foreground objects in the mask image are classified into regular and irregular class. For the fault separation of irregular targets such as pedestrians and animals,the vertical seed growth algorithm is designed in the target region. For the foreground targets of rules such as car and ships,the foreground seed in the design region is vertically and horizontally connected to remove the holes and the impact of the lack of structural information. Result The foreground target extraction is highly robust in complex environment with challenging interference factors.In the four groups of classic video of the database and the two videos of Shanxi Taichang Expressway,the dynamic background has the flow of water,swaying leaves,the slight jitter of the camera,and the change of light shadows. In addition,the experimental results were analyzed from three perspectives of the application effect,the accuracy of foreground target location,and the integrity of foreground target detection. The significance of target extraction algorithm has achieved an average accuracy of 90. 1%,an average recall of 88. 7%,and an average F value of 89. 4%,which are all superior to other similar algorithms. Compared with the mixed Gaussian model and the optical flow algorithm,the complex background brings a large noise disturbance. The Gaussian mixture model uses a morphological algorithm to remove the noise filling holes,giving the detected foreground target more viscous information. At different shadow area,the detection effect varies greatly.Furthermore,the optical flow algorithm is sensitive to light,and the changed light is mistaken for optical flow,which is not suitable under strict environmental requirements. Conclusion In this paper,by quickly locating the salient foreground,a parallel seed identification and optimized connection algorithm for RPCA initial screening image is proposed. The qualitative and quantitative analyses of the experimental data show that the algorithm can separate the foreground target from the dynamic background more quickly,reduce the adhesion between the foreground object and the background,and more effectively retain the structural information of the foreground object in the original image. In the following studies,deficiencies in the overall model and the algorithm details are continuously optimized. In the face of abnormal light rays,shadow suppression can be combined to make it more robust,and the performance and effectiveness of the algorithm are improved in more complex environments such as drone mobile video,which provides data support for feature extraction and abnormal behavior analysis.
引文
[1]Guo D J.Research on foreground extraction and targets detecting and tracking for video surveillance[D].Hangzhou:Zhejiang University,2016.[郭达洁.监控视频中的前景提取和目标检测跟踪算法研究[D].杭州:浙江大学,2016.]
[2]Li Q.Research on anomaly detection algorithm in video surveillance[D].Hefei:University of Science and Technology of China,2017.[李强.监控视频异常行为检测算法研究[D].合肥:中国科学技术大学,2017.]
[3]Araki S,Matsuoka T,Yokoya N,et al.Real-time tracking of multiple moving object contours in a moving camera image sequence[J].IEICE Transactions on Information and Systems,2000,E83-D(7):1583-1591.
[4]Hsieh J W.Fast stitching algorithm for moving object detection and mosaic construction[J].Image and Vision Computing,2004,22(4):291-306.[DOI:0.1016/j.imavis.2003.09.018]
[5]Tissainayagam P,Suter D.Object tracking in image sequences using point features[J].Pattern Recognition,2005,38(1):105-113.[DOI:10.1016/j.patcog.2004.05.011]
[6]Karmann K P.Moving object recognition using an adaptive background memory[M]//Cappellini V.Time-Varying Image Processing and Moving Object Recognition.Amsterdam,the Netherlands:Elsevier,1990:289-307.
[7]Kilger M.A shadow handler in a video-based real-time traffic monitoring system[C]//Proceedings of IEEE Workshop on Applications of Computer Vision.Palm Springs,CA,USA:IEEE,1992:11-18.[DOI:10.1109/ACV.1992.240332]
[8]Arras K O,Lau B,Grzonka S,et al.Range-based people detection and tracking for socially enabled service robots[M]//Prassler E,Z9llner M,Bischoff R,et al.Towards Service Robots for Everyday Environments.Berlin,Heidelberg:Springer,2012,76:235-280.[DOI:10.1007/978-3-642-25116-0_18]
[9]Li B C,Ding K.An improved algorithm of Gaussian mixture model combined with shadow suppression[J].Computer Engineering&Science,2016,38(3):556-561.[李博川,丁轲.结合阴影抑制的混合高斯模型改进算法[J].计算机工程与科学,2016,38(3):556-561.]
[10]Stauffer C,Grimson W E L.Adaptive background mixture models for real-time tracking[M]//Proceedings.1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Fort Collins,Colorado:IEEE Computer.Soc.,1999.
[11]Horn B K P,Schunck B G.Determining optic flow[J].International Journal of Artificial Intelligence,1981:185-203.
[12]Lucas B D,Kanade T.An iterative image registration technique with an application to stereo vision[C]//Proceedings of the 7th International Joint Conference on Artificial Intelligence.Vancouver,BC,Canada:Morgan Kaufmann Publishers Inc,1981:674-679.
[13]Li T.Research on methods of multiple objects tracking in intelligent visual surveillance[D].Hefei:University of Science and Technology of China,2013.[李彤.智能视频监控下的多目标跟踪技术研究[D].合肥:中国科学技术大学,2013.]
[14]Candès E J,Li X D,Ma Y,et al.Robust principal component analysis?[J].Journal of the ACM,2011,58(3):#11.[DOI:10.1145/1970392.1970395]
[15]Bao B K,Liu G C,Xu C S,et al.Inductive robust principal component analysis[J].IEEE Transactions on Image Processing,2012,21(8):3794-3800.[DOI:10.1109/TIP.2012.2192742]
[16]Gao Z,Cheong L F,Wang Y X.Block-sparse RPCA for salient motion detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(10):1975-1987.[DOI:10.1109/TPAMI.2014.2314663]
[17]Yang B,Zou L.Robust foreground detection using block-based RPCA[J].Optik-International Journal for Light and Electron Optics,2015,126(23):4586-4590.[DOI:10.1016/j.ijleo.2015.08.064]
[18]Zhou W,Sun Y B,Liu Q S,et al.l0group sparse RPCA model and algorithm for moving object detection[J].Acta Electronica Sinica,2016,44(3):627-632.[周伟,孙玉宝,刘青山,等.运动目标检测的l0群稀疏RPCA模型及其算法[J].电子学报,2016,44(3):627-632.][DOI:10.3969/j.issn.0372-2112.2016.03.020]
[19]Tao D.Research and implementation of key frame extraction in content based video retrieval system[D].Changchun:Jilin University,2004.[陶丹.基于内容的视频检索系统中关键帧提取方法的研究与实现[D].长春:吉林大学,2004.]
[20]Wen J J.Research on moving object detection and action analysis[D].Harbin:Harbin Institute of Technology,2015.[文嘉俊.运动目标检测及其行为分析研究[D].哈尔滨:哈尔滨工业大学,2015.]
[21]Peng B,Jiang Y S,Chen C,et al.Automatic parallel cracking detection algorithm based on 1 mm resolution 3D pavement images[J].Journal of Southeast University:Natural Science Edition,2015,45(6):1190-1196.[彭博,蒋阳升,陈成,等.基于1mm精度路面三维图像的裂缝自动并行识别算法[J].东南大学学报:自然科学版,2015,45(6):1190-1196.][DOI:10.3969/j.issn.1001-0505.2015.06.030]
[22]Gavilán M,Balcones D,Marcos O,et al.Adaptive road crack detection system by pavement classification[J].Sensors,2011,11(10):9628-9657.[DOI:10.3390/s111009628]
[23]Zhang D J,Li Q Q,Chen Y,et al.An efficient and reliable coarse-to-fine approach for asphalt pavement crack detection[J].Image and Vision Computing,2017,57:130-146.[DOI:10.1016/j.imavis.2016.11.018]
[24]Kang Z,Peng C,Cheng Q.Robust PCA via nonconvex rank approximation[C]//Proceedings of 2015 IEEE International Conference on Data Mining.Atlantic City,NJ,USA:IEEE,2015:211-220.[DOI:10.1109/ICDM.2015.15]
[25]Luo Q,Han Z,Chen X A,et al.Tensor RPCA by bayesian CPfactorization with complex noise[C]//Proceedings of 2017 IEEEInternational Conference on Computer Vision.Venice,Italy:IEEE,2017:5029-5038.
[26]Zhou T,Tao D.Unmixing Incoherent Structures of Big Data by Randomized or Greedy Decomposition[J].Computer Science,2013.
[27]Chen L,Zhang R G,Hu J,et al.Improved Gaussian mixture model and shadow elimination method[J].Journal of Computer Applications,2013,33(5):1394-1397.[DOI:10.3724/SP.J.1087.2013.01394]
[28]Tang Z,Miao Z J.Fast background subtraction and shadow elimination using improved Gaussian mixture model[C]//Proceedings of 2007 IEEE International Workshop on Haptic,Audio and Visual Environments and Games.Ottawa,Ont.,Canada:IEEE,2007:38-41.[DOI:10.1109/HAVE.2007.4371583]
[29]Matthews I,Baker S.Active appearance models revisited[J].International Journal of Computer Vision,2004,60(2):135-164.[DOI:10.1023/B:VISI.0000029666.37597.d3]
[30]Zou Q,Cao Y,Li Q Q,et al.CrackTree:automatic crack detection from pavement images[J].Pattern Recognition Letters,2012,33(3):227-238.[DOI:10.1016/j.patrec.2011.11.004]
[31]Li Q Q,Zou Q,Liu X L.Pavement crack classification via spatial distribution features[J].EURASIP Journal on Advances in Signal Processing,2011,2011:#649675.
[2]Li Q.Research on anomaly detection algorithm in video surveillance[D].Hefei:University of Science and Technology of China,2017.[李强.监控视频异常行为检测算法研究[D].合肥:中国科学技术大学,2017.]
[3]Araki S,Matsuoka T,Yokoya N,et al.Real-time tracking of multiple moving object contours in a moving camera image sequence[J].IEICE Transactions on Information and Systems,2000,E83-D(7):1583-1591.
[4]Hsieh J W.Fast stitching algorithm for moving object detection and mosaic construction[J].Image and Vision Computing,2004,22(4):291-306.[DOI:0.1016/j.imavis.2003.09.018]
[5]Tissainayagam P,Suter D.Object tracking in image sequences using point features[J].Pattern Recognition,2005,38(1):105-113.[DOI:10.1016/j.patcog.2004.05.011]
[6]Karmann K P.Moving object recognition using an adaptive background memory[M]//Cappellini V.Time-Varying Image Processing and Moving Object Recognition.Amsterdam,the Netherlands:Elsevier,1990:289-307.
[7]Kilger M.A shadow handler in a video-based real-time traffic monitoring system[C]//Proceedings of IEEE Workshop on Applications of Computer Vision.Palm Springs,CA,USA:IEEE,1992:11-18.[DOI:10.1109/ACV.1992.240332]
[8]Arras K O,Lau B,Grzonka S,et al.Range-based people detection and tracking for socially enabled service robots[M]//Prassler E,Z9llner M,Bischoff R,et al.Towards Service Robots for Everyday Environments.Berlin,Heidelberg:Springer,2012,76:235-280.[DOI:10.1007/978-3-642-25116-0_18]
[9]Li B C,Ding K.An improved algorithm of Gaussian mixture model combined with shadow suppression[J].Computer Engineering&Science,2016,38(3):556-561.[李博川,丁轲.结合阴影抑制的混合高斯模型改进算法[J].计算机工程与科学,2016,38(3):556-561.]
[10]Stauffer C,Grimson W E L.Adaptive background mixture models for real-time tracking[M]//Proceedings.1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Fort Collins,Colorado:IEEE Computer.Soc.,1999.
[11]Horn B K P,Schunck B G.Determining optic flow[J].International Journal of Artificial Intelligence,1981:185-203.
[12]Lucas B D,Kanade T.An iterative image registration technique with an application to stereo vision[C]//Proceedings of the 7th International Joint Conference on Artificial Intelligence.Vancouver,BC,Canada:Morgan Kaufmann Publishers Inc,1981:674-679.
[13]Li T.Research on methods of multiple objects tracking in intelligent visual surveillance[D].Hefei:University of Science and Technology of China,2013.[李彤.智能视频监控下的多目标跟踪技术研究[D].合肥:中国科学技术大学,2013.]
[14]Candès E J,Li X D,Ma Y,et al.Robust principal component analysis?[J].Journal of the ACM,2011,58(3):#11.[DOI:10.1145/1970392.1970395]
[15]Bao B K,Liu G C,Xu C S,et al.Inductive robust principal component analysis[J].IEEE Transactions on Image Processing,2012,21(8):3794-3800.[DOI:10.1109/TIP.2012.2192742]
[16]Gao Z,Cheong L F,Wang Y X.Block-sparse RPCA for salient motion detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(10):1975-1987.[DOI:10.1109/TPAMI.2014.2314663]
[17]Yang B,Zou L.Robust foreground detection using block-based RPCA[J].Optik-International Journal for Light and Electron Optics,2015,126(23):4586-4590.[DOI:10.1016/j.ijleo.2015.08.064]
[18]Zhou W,Sun Y B,Liu Q S,et al.l0group sparse RPCA model and algorithm for moving object detection[J].Acta Electronica Sinica,2016,44(3):627-632.[周伟,孙玉宝,刘青山,等.运动目标检测的l0群稀疏RPCA模型及其算法[J].电子学报,2016,44(3):627-632.][DOI:10.3969/j.issn.0372-2112.2016.03.020]
[19]Tao D.Research and implementation of key frame extraction in content based video retrieval system[D].Changchun:Jilin University,2004.[陶丹.基于内容的视频检索系统中关键帧提取方法的研究与实现[D].长春:吉林大学,2004.]
[20]Wen J J.Research on moving object detection and action analysis[D].Harbin:Harbin Institute of Technology,2015.[文嘉俊.运动目标检测及其行为分析研究[D].哈尔滨:哈尔滨工业大学,2015.]
[21]Peng B,Jiang Y S,Chen C,et al.Automatic parallel cracking detection algorithm based on 1 mm resolution 3D pavement images[J].Journal of Southeast University:Natural Science Edition,2015,45(6):1190-1196.[彭博,蒋阳升,陈成,等.基于1mm精度路面三维图像的裂缝自动并行识别算法[J].东南大学学报:自然科学版,2015,45(6):1190-1196.][DOI:10.3969/j.issn.1001-0505.2015.06.030]
[22]Gavilán M,Balcones D,Marcos O,et al.Adaptive road crack detection system by pavement classification[J].Sensors,2011,11(10):9628-9657.[DOI:10.3390/s111009628]
[23]Zhang D J,Li Q Q,Chen Y,et al.An efficient and reliable coarse-to-fine approach for asphalt pavement crack detection[J].Image and Vision Computing,2017,57:130-146.[DOI:10.1016/j.imavis.2016.11.018]
[24]Kang Z,Peng C,Cheng Q.Robust PCA via nonconvex rank approximation[C]//Proceedings of 2015 IEEE International Conference on Data Mining.Atlantic City,NJ,USA:IEEE,2015:211-220.[DOI:10.1109/ICDM.2015.15]
[25]Luo Q,Han Z,Chen X A,et al.Tensor RPCA by bayesian CPfactorization with complex noise[C]//Proceedings of 2017 IEEEInternational Conference on Computer Vision.Venice,Italy:IEEE,2017:5029-5038.
[26]Zhou T,Tao D.Unmixing Incoherent Structures of Big Data by Randomized or Greedy Decomposition[J].Computer Science,2013.
[27]Chen L,Zhang R G,Hu J,et al.Improved Gaussian mixture model and shadow elimination method[J].Journal of Computer Applications,2013,33(5):1394-1397.[DOI:10.3724/SP.J.1087.2013.01394]
[28]Tang Z,Miao Z J.Fast background subtraction and shadow elimination using improved Gaussian mixture model[C]//Proceedings of 2007 IEEE International Workshop on Haptic,Audio and Visual Environments and Games.Ottawa,Ont.,Canada:IEEE,2007:38-41.[DOI:10.1109/HAVE.2007.4371583]
[29]Matthews I,Baker S.Active appearance models revisited[J].International Journal of Computer Vision,2004,60(2):135-164.[DOI:10.1023/B:VISI.0000029666.37597.d3]
[30]Zou Q,Cao Y,Li Q Q,et al.CrackTree:automatic crack detection from pavement images[J].Pattern Recognition Letters,2012,33(3):227-238.[DOI:10.1016/j.patrec.2011.11.004]
[31]Li Q Q,Zou Q,Liu X L.Pavement crack classification via spatial distribution features[J].EURASIP Journal on Advances in Signal Processing,2011,2011:#649675.