基于视频的室内异常行为分析方法的研究
|
摘要
基于视觉的人体行为分析是当前一个非常活跃的研究领域,而对人的行为进行理解和分析则是该领域内富有挑战性的研究方向,它在智能监控、感知接口和基于内容的视频检索等领域具有广泛的应用前景。目前,人体异常行为的研究主要针对单人典型行为进行,而对单人潜在的异常行为及多人冲突行为的研究较少,而且行为分析还没有一个系统的理论框架。作为行为分析基础的运动目标检测算法因受光线变化等噪声的影响比较大,导致行为分析的效果不理想。本学位论文提出了利用人体上半身质心轨迹分析室内异常行为的方法,并提出了利用能量来分析两人间冲突行为以及利用颜色模型来检测吸烟行为的方法。
首先,在室内环境下,通过自适应背景更新方法提取出运动目标并划分连通区域,根据连通区域的个数将该场景划为单人或多人。在单人场景下,提取运动目标上半身质心运动轨迹,根据其周期特征分析该人行为是否正常。
其次,在多人无遮挡场景下,通过R变换识别个体动作,从而判别异常动作。在多人相互遮挡的场景下提取运动目标的运动能量,根据其能量图分析是否发生冲突行为。
最后,在特定场景下,通过肤色模型判断场景中是否存在人体,在有人的前提下,结合火焰在时域中亮度瞬间变化和烟雾由下至上的运动特征,判断该场景中是否存在吸烟行为。
仿真实验结果表明,基于人体上半身质心运动轨迹的行走状态分析方法能够准确分析室内行走的运动状态,识别率达到92%,并且对光线变化、背景噪声具有良好的鲁棒性。运动能量能较好地识别单人潜在的异常动作及多人冲突行为,识别率分别达到84%和80%。结合多特征的吸烟行为检测方法,能够较好地判断场景中是否存在吸烟行为。
Vision-based human behavior analysis is currently one of the most active research fields, and to understand and analyze the human behavior is a challenging research topic. Human behavior analysis has many promising applications such as intelligent surveillance, perceptual interface and content-based video retrieval. Most research on human behavior analysis is based on single person and simple action; however, there is less research about single potential abnormal behavior and multi-person fighting behavior. As the basic of the human behavior analysis, the result of human behavior analysis always gets poor effect because of the noise in motion detection, such as varying luminance. In this dissertation, an approach to analyzing person states based on multi-viewing-angle was proposed, and then, a method to analyzing fighting behavior based on energy and to detect smoking based on color model were also proposed.
Firstly, the moving person was detected using self-adaptive background updating algorithm and scene was estimated by the means of connected regions. After searching the centroid of upper part of the body, its periodic feature was extracted and the behavior was analyzed in single person scene.
Secondly, human motion was discriminated by R transform in scene where there was no occlusion, and then fighting behavior was analyzed based on energy image in multi-person scene.
Finally, in a given scene, person was detected by detecting skin area. And smoking was detected and analyzed by varying luminance, color model and directional information.
The experimental results show that the human speed and walking states method is reasonably robust in varying luminance and shadow, and the accuracy of walking states is 92%. Moving energy can be used to recognize abnormal behavior, and the accuracy is 84% and 80% in single person and multi-person scene respectively. Smoking behaviors can be exactly analysis by multi features.
首先,在室内环境下,通过自适应背景更新方法提取出运动目标并划分连通区域,根据连通区域的个数将该场景划为单人或多人。在单人场景下,提取运动目标上半身质心运动轨迹,根据其周期特征分析该人行为是否正常。
其次,在多人无遮挡场景下,通过R变换识别个体动作,从而判别异常动作。在多人相互遮挡的场景下提取运动目标的运动能量,根据其能量图分析是否发生冲突行为。
最后,在特定场景下,通过肤色模型判断场景中是否存在人体,在有人的前提下,结合火焰在时域中亮度瞬间变化和烟雾由下至上的运动特征,判断该场景中是否存在吸烟行为。
仿真实验结果表明,基于人体上半身质心运动轨迹的行走状态分析方法能够准确分析室内行走的运动状态,识别率达到92%,并且对光线变化、背景噪声具有良好的鲁棒性。运动能量能较好地识别单人潜在的异常动作及多人冲突行为,识别率分别达到84%和80%。结合多特征的吸烟行为检测方法,能够较好地判断场景中是否存在吸烟行为。
Vision-based human behavior analysis is currently one of the most active research fields, and to understand and analyze the human behavior is a challenging research topic. Human behavior analysis has many promising applications such as intelligent surveillance, perceptual interface and content-based video retrieval. Most research on human behavior analysis is based on single person and simple action; however, there is less research about single potential abnormal behavior and multi-person fighting behavior. As the basic of the human behavior analysis, the result of human behavior analysis always gets poor effect because of the noise in motion detection, such as varying luminance. In this dissertation, an approach to analyzing person states based on multi-viewing-angle was proposed, and then, a method to analyzing fighting behavior based on energy and to detect smoking based on color model were also proposed.
Firstly, the moving person was detected using self-adaptive background updating algorithm and scene was estimated by the means of connected regions. After searching the centroid of upper part of the body, its periodic feature was extracted and the behavior was analyzed in single person scene.
Secondly, human motion was discriminated by R transform in scene where there was no occlusion, and then fighting behavior was analyzed based on energy image in multi-person scene.
Finally, in a given scene, person was detected by detecting skin area. And smoking was detected and analyzed by varying luminance, color model and directional information.
The experimental results show that the human speed and walking states method is reasonably robust in varying luminance and shadow, and the accuracy of walking states is 92%. Moving energy can be used to recognize abnormal behavior, and the accuracy is 84% and 80% in single person and multi-person scene respectively. Smoking behaviors can be exactly analysis by multi features.
引文
1 徐光祐,曹媛媛.动作识别与行为理解综述.中国图像图形学报.2009,14(2)：189-195
2 肖伯祥,张强,魏小鹏.人体运动捕捉数据特征提取与检索研究综述.计算机应用研究.2010,27(1)：10-12
3 J. K. Aggarwal, S. Park. Human Motion: Modeling and Recognition of Actions and Interactions. Second International Symposium on 3D Data Processing. Texas, USA,2004:640～647
4 J. J. Gibson. The Perception of the Visual World. Boston:Houghton Mifflin, 1950:174～179
5 杨国亮,王志良,牟世堂等.一种改进的光流算法.计算机工程.2006,32(1 5)：187-226
6 于慧敏,尤育赛.基于水平集的多运动目标检测和分割.浙江大学学报.2007,41(3)：412-417
7 侯志强,韩崇昭.基于像素灰度归类的背景重构方法.软件学报.2005,16(9)：1568-1576
8 刘扬,黄庆明,高文等.自适应高斯混合模型球场检测算法及其在体育视频分析中的应用.计算机研究与发展.2006,43(7)：1207-1215
9 敖友云,迟洪钦.多目标差分演化算法研究综述.计算机科学与探索.2009,3(3)：234-246
10 孙志海,朱善安.多视频运动对象实时分割及跟踪技术.浙江大学学报.2008,42(9)：1631-1635
11 T. Yang, S. Z. Li, Q. Pan et al. Real-Time and Accurate Segmentation of Moving Objects in Dynamic Scene. IEEE Transactions on Circuits and Systems for Video Technology.2004,14(6):796～812
12 郑可飚,黄文清,张佐理.运动目标跟踪系统的遮挡问题处理.计算机工程与设计.2009,30(11)：2816-2818
13 梁华,刘云辉.视频中遮挡情况下目标的跟踪.计算机应用研究.2008,25(3)：950-952,956
14 F. T. Luis, C. R. Luis. Video Object Matching Across Multiple Independent Views Using Local Descriptors and Adaptive Learning. Pattern Recognition Letters.2009,30(2):157～167
15 H. Y. Cheng, J. N. Hwang. Adaptive Particle Sampling and Adaptive Appearance for Multiple Video Object Tracking. Signal Processing.2009, 89(9):1844～1849
16 A. Colombari, A. Fusiello, V. Murino. Segmentation and Tracking of Multiple Video Objects. The Journal of the Pattern Recognition Society. 2007,40(4):1307～1317
17 刘少华,张茂军,熊志辉等.一种鲁棒高效的视频运动目标检测与跟踪算法.自动化学报.2009,35(8)：1055-1062
18 万琴,王耀南.一种多运动目标检测、跟踪方法研究与实现.计算机应用研究.2007,24(1)：199-202
19 P. Spagnolo, T. D'Orazio, M. Leo et al. Moving Object Segmentation by Background Subtraction and Temporal Analysis. Image and Vision Computing.2006,24(5):411～423
20 钱惠敏,茅耀斌,王执铨等.视频监控中的行为序列分割与识别.中国图象图形学报.2009,14(11)：2416-2420
21 万卫兵,霍宏,赵志明.智能视频监控中目标检测与识别.上海：上海交通大学出版社,2010：10-12
22 张军,刘志镜.可变场所的异常行为识别方法.中国图象图形学报.2009,14(10)：2097-2101
23 T. Huynh, B. Schiele. Analyzing Features for Activity Recognition. sOc-EUSAI'05. New York, USA.2005:159～164
24 R. Liu, S. Gregory, H. Jessica et al. Learning Silhouette Features for Control of Human Motion. International Conference on Computer Graphics and Interactive Techniques.2004:1303～1330
25 曹媛媛,黄飞跃,陶霖密.自然环境下日常动作的在线识别.电子学报.2009.4,37(4)：16-21
26 F. Martinez-Contreras, C. Orrite-Urunuela, E. Herrero-Jaraba et al. Recognizing Human Actions Using Silhouette-based HMM.6th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS. Genoa, Italy,2009:43～48
27 X. D. Sun, C. W. Chen, B.S. Manjunath. Probabilistic Motion Parameter Models for Human Activity Recognition. IEEE International Conference on Pattern Recognition. Quebec City, Canada,2002:443～446
28 C. Fernandez, P. Baiget, X. Roca, J. Gonzalez. Natural Language Descriptions of Human Behavior from Video Sequences.30th Annual German Conference on AI(KI), Osnabruck, Germany,2007:279～292
29 A. F. Bobick, J. W. Davis. The Recognition of Human Movement Using Temporal Templates. IEEE Transactions on Pattern Analysis and Machine Intelligence.2001,23(3):257～267
30 王科俊,侯本博.步态识别综述.中国图象图形学报.2007,12(7)：1152-1160
31 陈实,马天骏,高有行.用行人轮廓的分布直方图分类和识别步态.计算机研究与发展.2009,46(2)：295-301
32 K. Daehee, L. Seungwon, P. Jooki. Active Shape Model-Based on Gait Recognition Using Infrared Images. International Journal of Signal Processing, Image Processing and Pattern Recognition.2009,2(4):1～12
33 C. Huang, H. Z. Ai, Y. Li et al. Vector Boosting for Rotation Invariant Multi-View Face Detection. Computer Vision,2005. ICCV 2005. Tenth IEEE International Conference on. Beijing, China,2005:446～453
34 洪娥,徐海涛.一种基于链码技术的图像目标物体面积统计新方法.计算机应用研究.2008,25(1)：303-305
35 徐成,冯斌,刘彦.一种局部纹理特征在区域生长中的应用.计算机应用研究.2009,26(12)：4852-4854
36 S. Tabbone, L. Wendling, J. P. Salmon. A New Shape Descriptor Defined on the Radon Transform. Computer Vis. Image Understanding.2006, I02(1):42～51
37 R. C. Gonzalez, R. E. Woods数字图像处理.阮秋琦等译.第2版.北京：电子工业出版社,2007：228-239
38 R. L. Hsu, A. M. Mohamed, K. J. Anil. Face Detection in Color Image. IEEE Transactions on Pattern Analysis and Machine Intelligence.2002,24(5): 696～706
39 赵国英,向世明,李华.基于反射对称的步态序列识别.计算机辅助设计与图形学学报.2005.17(10)：2239-2244
40 Z. Szlavik, L. Kovacs.L. Havasi et al. Behavior and Event Detection for Annotation and Surveillance. Content-Based Multimedia Indexing, CBMI 2008. International Workshop on. London, England,2008:117～124
41 王文豪,刘尚勤,叶庆生.视频火灾检测的设计与实现.计算机工程与设计.2008,29(3)：775-778
42 许宏科,房建武,文常保.基于亮度与火焰区域边缘颜色分布的火焰检测.计算机应用研究.2010,27(9)：3581-3584
43 袁非牛,张永明,刘士兴等.基于累积量和主运动方向的视频烟雾检测方法.中国图象图形学报.2008,13(4)：808-813
44 C. C. Ho, T. H. Kuo. Real-Time Video-Based Fire Smoke Detection System. 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics. Singapore,2009:1845～1850
2 肖伯祥,张强,魏小鹏.人体运动捕捉数据特征提取与检索研究综述.计算机应用研究.2010,27(1)：10-12
3 J. K. Aggarwal, S. Park. Human Motion: Modeling and Recognition of Actions and Interactions. Second International Symposium on 3D Data Processing. Texas, USA,2004:640～647
4 J. J. Gibson. The Perception of the Visual World. Boston:Houghton Mifflin, 1950:174～179
5 杨国亮,王志良,牟世堂等.一种改进的光流算法.计算机工程.2006,32(1 5)：187-226
6 于慧敏,尤育赛.基于水平集的多运动目标检测和分割.浙江大学学报.2007,41(3)：412-417
7 侯志强,韩崇昭.基于像素灰度归类的背景重构方法.软件学报.2005,16(9)：1568-1576
8 刘扬,黄庆明,高文等.自适应高斯混合模型球场检测算法及其在体育视频分析中的应用.计算机研究与发展.2006,43(7)：1207-1215
9 敖友云,迟洪钦.多目标差分演化算法研究综述.计算机科学与探索.2009,3(3)：234-246
10 孙志海,朱善安.多视频运动对象实时分割及跟踪技术.浙江大学学报.2008,42(9)：1631-1635
11 T. Yang, S. Z. Li, Q. Pan et al. Real-Time and Accurate Segmentation of Moving Objects in Dynamic Scene. IEEE Transactions on Circuits and Systems for Video Technology.2004,14(6):796～812
12 郑可飚,黄文清,张佐理.运动目标跟踪系统的遮挡问题处理.计算机工程与设计.2009,30(11)：2816-2818
13 梁华,刘云辉.视频中遮挡情况下目标的跟踪.计算机应用研究.2008,25(3)：950-952,956
14 F. T. Luis, C. R. Luis. Video Object Matching Across Multiple Independent Views Using Local Descriptors and Adaptive Learning. Pattern Recognition Letters.2009,30(2):157～167
15 H. Y. Cheng, J. N. Hwang. Adaptive Particle Sampling and Adaptive Appearance for Multiple Video Object Tracking. Signal Processing.2009, 89(9):1844～1849
16 A. Colombari, A. Fusiello, V. Murino. Segmentation and Tracking of Multiple Video Objects. The Journal of the Pattern Recognition Society. 2007,40(4):1307～1317
17 刘少华,张茂军,熊志辉等.一种鲁棒高效的视频运动目标检测与跟踪算法.自动化学报.2009,35(8)：1055-1062
18 万琴,王耀南.一种多运动目标检测、跟踪方法研究与实现.计算机应用研究.2007,24(1)：199-202
19 P. Spagnolo, T. D'Orazio, M. Leo et al. Moving Object Segmentation by Background Subtraction and Temporal Analysis. Image and Vision Computing.2006,24(5):411～423
20 钱惠敏,茅耀斌,王执铨等.视频监控中的行为序列分割与识别.中国图象图形学报.2009,14(11)：2416-2420
21 万卫兵,霍宏,赵志明.智能视频监控中目标检测与识别.上海：上海交通大学出版社,2010：10-12
22 张军,刘志镜.可变场所的异常行为识别方法.中国图象图形学报.2009,14(10)：2097-2101
23 T. Huynh, B. Schiele. Analyzing Features for Activity Recognition. sOc-EUSAI'05. New York, USA.2005:159～164
24 R. Liu, S. Gregory, H. Jessica et al. Learning Silhouette Features for Control of Human Motion. International Conference on Computer Graphics and Interactive Techniques.2004:1303～1330
25 曹媛媛,黄飞跃,陶霖密.自然环境下日常动作的在线识别.电子学报.2009.4,37(4)：16-21
26 F. Martinez-Contreras, C. Orrite-Urunuela, E. Herrero-Jaraba et al. Recognizing Human Actions Using Silhouette-based HMM.6th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS. Genoa, Italy,2009:43～48
27 X. D. Sun, C. W. Chen, B.S. Manjunath. Probabilistic Motion Parameter Models for Human Activity Recognition. IEEE International Conference on Pattern Recognition. Quebec City, Canada,2002:443～446
28 C. Fernandez, P. Baiget, X. Roca, J. Gonzalez. Natural Language Descriptions of Human Behavior from Video Sequences.30th Annual German Conference on AI(KI), Osnabruck, Germany,2007:279～292
29 A. F. Bobick, J. W. Davis. The Recognition of Human Movement Using Temporal Templates. IEEE Transactions on Pattern Analysis and Machine Intelligence.2001,23(3):257～267
30 王科俊,侯本博.步态识别综述.中国图象图形学报.2007,12(7)：1152-1160
31 陈实,马天骏,高有行.用行人轮廓的分布直方图分类和识别步态.计算机研究与发展.2009,46(2)：295-301
32 K. Daehee, L. Seungwon, P. Jooki. Active Shape Model-Based on Gait Recognition Using Infrared Images. International Journal of Signal Processing, Image Processing and Pattern Recognition.2009,2(4):1～12
33 C. Huang, H. Z. Ai, Y. Li et al. Vector Boosting for Rotation Invariant Multi-View Face Detection. Computer Vision,2005. ICCV 2005. Tenth IEEE International Conference on. Beijing, China,2005:446～453
34 洪娥,徐海涛.一种基于链码技术的图像目标物体面积统计新方法.计算机应用研究.2008,25(1)：303-305
35 徐成,冯斌,刘彦.一种局部纹理特征在区域生长中的应用.计算机应用研究.2009,26(12)：4852-4854
36 S. Tabbone, L. Wendling, J. P. Salmon. A New Shape Descriptor Defined on the Radon Transform. Computer Vis. Image Understanding.2006, I02(1):42～51
37 R. C. Gonzalez, R. E. Woods数字图像处理.阮秋琦等译.第2版.北京：电子工业出版社,2007：228-239
38 R. L. Hsu, A. M. Mohamed, K. J. Anil. Face Detection in Color Image. IEEE Transactions on Pattern Analysis and Machine Intelligence.2002,24(5): 696～706
39 赵国英,向世明,李华.基于反射对称的步态序列识别.计算机辅助设计与图形学学报.2005.17(10)：2239-2244
40 Z. Szlavik, L. Kovacs.L. Havasi et al. Behavior and Event Detection for Annotation and Surveillance. Content-Based Multimedia Indexing, CBMI 2008. International Workshop on. London, England,2008:117～124
41 王文豪,刘尚勤,叶庆生.视频火灾检测的设计与实现.计算机工程与设计.2008,29(3)：775-778
42 许宏科,房建武,文常保.基于亮度与火焰区域边缘颜色分布的火焰检测.计算机应用研究.2010,27(9)：3581-3584
43 袁非牛,张永明,刘士兴等.基于累积量和主运动方向的视频烟雾检测方法.中国图象图形学报.2008,13(4)：808-813
44 C. C. Ho, T. H. Kuo. Real-Time Video-Based Fire Smoke Detection System. 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics. Singapore,2009:1845～1850