时空深度特征AP聚类的稀疏表示视频异常检测算法
|
摘要
针对异常行为检测问题,提出基于时空深度特征的AP聚类稀疏表示视频异常检测方法。由于视频序列中大量背景信息及有效信息分布不均匀的情况,首先利用光流结合非均匀的细胞分割对视频的运动目标进行提取并得到空间尺寸大小不同的时空兴趣块。其次利用三维卷积神经网络提取不同时空兴趣块的时空深度特征从而对原始视频序列进行三维描述。然后在字典学习时,采用AP聚类方法,将训练样本中具有代表性的特征作为字典,极大降低字典维度以及稀疏表示方法对计算内存的要求。本文将测试样本进行AP聚类后仅对具有代表性的聚类中心进行检测,在减少实验时间的同时削减了阈值对检测效果的敏感度。实验结果表明,与现有的检测方法相比本文方法具有优越性。
In this paper, we propose a novel sparse reconstruction cost based AP clustering and spatial-temporal convolution neural net to detect anomaly behavior in crowd sense of video sequence. Considering several background information in video and the position of camera, we use optical-flow method and grid with variable-sized cells for video frame to extract a great number of spatial temporal interest cuboid with different spatial sizes. 3D convolution neural network is used to obtain the spatial-temporal features of different spatial-temporal interest cuboid, so that the original video sequences can be described in three dimensions. In order to deal the large number of features of spatial-temporal interest cuboids, we apply the AP clustering method in dictionary learning, and the representative feature of the training sample is add to the dictionary, which greatly reduces the dictionary dimension and reduces the memory requirement of sparse representation. In the test stage, consider the similarity among normal samples, we apply sparse reconstruction cost with AP clustering to reduce the computational cost. Experimental result on challenging abnormality detection dataset show the advantage of the proposed method compared to the state-of-the-art in different abnormality detection tasks.
In this paper, we propose a novel sparse reconstruction cost based AP clustering and spatial-temporal convolution neural net to detect anomaly behavior in crowd sense of video sequence. Considering several background information in video and the position of camera, we use optical-flow method and grid with variable-sized cells for video frame to extract a great number of spatial temporal interest cuboid with different spatial sizes. 3D convolution neural network is used to obtain the spatial-temporal features of different spatial-temporal interest cuboid, so that the original video sequences can be described in three dimensions. In order to deal the large number of features of spatial-temporal interest cuboids, we apply the AP clustering method in dictionary learning, and the representative feature of the training sample is add to the dictionary, which greatly reduces the dictionary dimension and reduces the memory requirement of sparse representation. In the test stage, consider the similarity among normal samples, we apply sparse reconstruction cost with AP clustering to reduce the computational cost. Experimental result on challenging abnormality detection dataset show the advantage of the proposed method compared to the state-of-the-art in different abnormality detection tasks.
引文
[1] Wang Siqi,Zhu En,Yin Jianping,et al.Video anomaly detection and localization by local motion based joint video representation and OCELM[J].Neurocomputing,2018,277(2):161-175.
[2] Biswas S,Venkatesh B R.Anomaly detection via short local trajectories[J].Neurocomputing,2017,242(6):63-72.
[3] Hajananth N,Fookes C,Denman S,et al.An mrf based abnormal event detection approach using motion and appearance features[C]//In Advanced Video and Signal Based Surveillance(AVSS),Seoul,South Korea:IEEE,2014:343-348.
[4] Liu Peng,Tao Ye,Zhao Wei,et al.Abnormal crowd motion detection using double sparse representation[J].Neurocomputing,2017,269(12):3-12.
[5] Yang Yanhua,Deng Cheng,Gao Shangqian,et al.Discriminative Multi-instance Multitask Learning for 3D Action Recognition[J].IEEE Transactions on Multimedia,2017,19(3):519-529.
[6] Yang Yanhua,Deng Cheng,Tao Dapeng,et al.Latent Max-margin Multitask Learning with Skelets for 3-D Action Recognition[J].IEEE Transactions on Cybernetics,2017,47(2):439- 448.
[7] Tran D,Bourdev L,Fergus R,et al.Learning Spatiotemporal Features with 3D Convolutional Networks[C]//IEEE International Conference on Computer Vision(ICCV),Santiago,Chile:IEEE,2015:4489- 4497.
[8] Zhou Shifu,Shen Wei,Zeng Dan,et al.Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes[J].Signal Processing:Image Communication,2016,47(9):358-368.
[9] Mohammad S,Mohsen F,Mahmood F,et al.Deep-Cascade:Cascading 3D Deep Neural Networks for Fast Anomaly Detection and Localization in Crowded Scenes[J].IEEE Transactions on Image Processing,2017,26(4):1992-2004.
[10] Ravanbakhsh M,Nabi M,Sangineto E,et al.Abnormal event detection in video using generative adversarial nets[C]//IEEE International Conference on Image Processing(ICIP),Beijing,China:IEEE,2017:1577-1281.
[11] Leyva R,Sanchez V,Li Chang-Tsun.Abnormal event detection in videos using binary features[C]//IEEE Telecommunication and Signal Processing(TSP),Barcelona,Spain:IEEE,2017:621- 625.
[12] Mohammad S,Mahmood F,Mojtaba H,et al.Real-Time Anomaly Detection and Localization in Crowded Scenes[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops,CVPRW 2015 IEEE Computer Society,Boston,MA,USA:IEEE,2015:56- 62.
[13] Simonyan K,Zisserman A.Two-Stream Convolutional Networks for Action Recognition in Videos[C]//Annual Conference on Neural Information Processing Systems,NIPS 2014 Montreal,QC,Canada:CVPR,2014:568-576.
[14] Frey B J,Dueck D.Clustering by passing messages between data points[J].Science,2007,315(5814):972-976.
[15] University of California,San Diego.UCSD Anomaly Detection Dataset[EB/OL].http://www.svcl.ucsd.edu/project-s/anomaly/dataset.html.2010-10-10.
[16] University of Minnesota.UMN abnormal events detection dataset[EB/OL].http://mha.cs.umn.deu/proj_events.shtml.2009- 4-12.
[17] Leyva R,Sanchez V,Li Chang-Tsun.Video Anomaly Detection With Compact Feature Sets for Online Performance[J].IEEE Transactions on Image Processing,2017,26(7):3463-3478.
[18] Mehran R,Oyama A,Shah M.Abnormal Crowd Behavior Detection using Social Force Model[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Miami,FL,United states:IEEE,2009:935-942.
[19] Adam A,Rivlin E,Shimshoni I,et al.Robust real-time unusual event detection using multiple fixed-location monitors[J].IEEE Transaction on Pattern Analysis and Machine Intelligence,2008,30(3):555-560.
[20] Mahadevan V,Li Weixin,Bhalodia V,et al.Anomaly detection in crowded scenes[J].Computer Vision & Pattern Recognition,2010,26(2):1975-1981.
[2] Biswas S,Venkatesh B R.Anomaly detection via short local trajectories[J].Neurocomputing,2017,242(6):63-72.
[3] Hajananth N,Fookes C,Denman S,et al.An mrf based abnormal event detection approach using motion and appearance features[C]//In Advanced Video and Signal Based Surveillance(AVSS),Seoul,South Korea:IEEE,2014:343-348.
[4] Liu Peng,Tao Ye,Zhao Wei,et al.Abnormal crowd motion detection using double sparse representation[J].Neurocomputing,2017,269(12):3-12.
[5] Yang Yanhua,Deng Cheng,Gao Shangqian,et al.Discriminative Multi-instance Multitask Learning for 3D Action Recognition[J].IEEE Transactions on Multimedia,2017,19(3):519-529.
[6] Yang Yanhua,Deng Cheng,Tao Dapeng,et al.Latent Max-margin Multitask Learning with Skelets for 3-D Action Recognition[J].IEEE Transactions on Cybernetics,2017,47(2):439- 448.
[7] Tran D,Bourdev L,Fergus R,et al.Learning Spatiotemporal Features with 3D Convolutional Networks[C]//IEEE International Conference on Computer Vision(ICCV),Santiago,Chile:IEEE,2015:4489- 4497.
[8] Zhou Shifu,Shen Wei,Zeng Dan,et al.Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes[J].Signal Processing:Image Communication,2016,47(9):358-368.
[9] Mohammad S,Mohsen F,Mahmood F,et al.Deep-Cascade:Cascading 3D Deep Neural Networks for Fast Anomaly Detection and Localization in Crowded Scenes[J].IEEE Transactions on Image Processing,2017,26(4):1992-2004.
[10] Ravanbakhsh M,Nabi M,Sangineto E,et al.Abnormal event detection in video using generative adversarial nets[C]//IEEE International Conference on Image Processing(ICIP),Beijing,China:IEEE,2017:1577-1281.
[11] Leyva R,Sanchez V,Li Chang-Tsun.Abnormal event detection in videos using binary features[C]//IEEE Telecommunication and Signal Processing(TSP),Barcelona,Spain:IEEE,2017:621- 625.
[12] Mohammad S,Mahmood F,Mojtaba H,et al.Real-Time Anomaly Detection and Localization in Crowded Scenes[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops,CVPRW 2015 IEEE Computer Society,Boston,MA,USA:IEEE,2015:56- 62.
[13] Simonyan K,Zisserman A.Two-Stream Convolutional Networks for Action Recognition in Videos[C]//Annual Conference on Neural Information Processing Systems,NIPS 2014 Montreal,QC,Canada:CVPR,2014:568-576.
[14] Frey B J,Dueck D.Clustering by passing messages between data points[J].Science,2007,315(5814):972-976.
[15] University of California,San Diego.UCSD Anomaly Detection Dataset[EB/OL].http://www.svcl.ucsd.edu/project-s/anomaly/dataset.html.2010-10-10.
[16] University of Minnesota.UMN abnormal events detection dataset[EB/OL].http://mha.cs.umn.deu/proj_events.shtml.2009- 4-12.
[17] Leyva R,Sanchez V,Li Chang-Tsun.Video Anomaly Detection With Compact Feature Sets for Online Performance[J].IEEE Transactions on Image Processing,2017,26(7):3463-3478.
[18] Mehran R,Oyama A,Shah M.Abnormal Crowd Behavior Detection using Social Force Model[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Miami,FL,United states:IEEE,2009:935-942.
[19] Adam A,Rivlin E,Shimshoni I,et al.Robust real-time unusual event detection using multiple fixed-location monitors[J].IEEE Transaction on Pattern Analysis and Machine Intelligence,2008,30(3):555-560.
[20] Mahadevan V,Li Weixin,Bhalodia V,et al.Anomaly detection in crowded scenes[J].Computer Vision & Pattern Recognition,2010,26(2):1975-1981.