基于FPGA的运动目标检测系统的研究与开发
摘要
视频序列中运动目标的检测是计算机视觉和图像编码研究领域的一个重要课题,在机器人导航、智能监视系统、交通监测、医学图像处理以及视频图像压缩和传输等领域都有广泛的应用。FPGA作为当今主流的大规模可编程专用集成电路,可以满足高速图像处理的需要。使用FPGA可以充分利用硬件上的并行性,从本质上改善图像处理的速度,使对大数据量的图像处理达到实时性。本文提出基于FPGA的运动目标检测系统,对以后算法的改进,输入输出图像大小的变化,图像采集和显示设备更换等都具有灵活性。
本文对目前运动目标检测的主要算法研究分析,根据背景减法的适用环境和特点提出改进的W4运动检测算法。该算法具备背景减法的优点,并且克服了W4运动检测算法在环境变化较快或环境变化较频繁条件下对运动目标进行检测的局限性。
本文首先在MATLAB中对改进的W4运动检测算法进行仿真,然后将算法移植到FPGA中实现。设计图像采集、图像检测和VGA显示等模块,完善运动目标检测系统。根据算法和运动目标检测系统的特点提出一种基于改进的W4算法的快速检测方法,该方法以块为单位进行运动目标检测,可以有效地提高图像处理的速度,使系统满足实时性要求。
Motion objects detection is an important problem of computer vision and image coding. It is widely used in automaton navigation, intelligent video surveillance systems, traffic detection, medical image processing, video compression and transmission and so on. As a main large scale programmable specific integrated circuit, FPGA meets the requirements of high speed image processing, with the characteristics of simple algorithm realization, easy programming, good portability and inheritability. In this thesis, we develop a motion detection system based on FPGA. It is flexible for later changing of algorithm, image size and collection and display equipments.
By analyzing some of the main algorithm of motion detection, especially background subtraction, a new motion detection algorithm called improved W4 is presented. The novel algorithm has the advantages of background subtraction and overcomes the shortages of W4 algorithm that can not efficiently detect moving object in frequent varying environmental weather condition.
We use MATLAB to simulate the improved W4 algorithm, and then port it to the FPGA. We design image collection, motion detection and display modules to consummate the motion detection system. In order to accelerate it to be a real-time system, a fast algorithm which uses block instead of pixel in improved W4 algorithm is presented.
本文对目前运动目标检测的主要算法研究分析,根据背景减法的适用环境和特点提出改进的W4运动检测算法。该算法具备背景减法的优点,并且克服了W4运动检测算法在环境变化较快或环境变化较频繁条件下对运动目标进行检测的局限性。
本文首先在MATLAB中对改进的W4运动检测算法进行仿真,然后将算法移植到FPGA中实现。设计图像采集、图像检测和VGA显示等模块,完善运动目标检测系统。根据算法和运动目标检测系统的特点提出一种基于改进的W4算法的快速检测方法,该方法以块为单位进行运动目标检测,可以有效地提高图像处理的速度,使系统满足实时性要求。
Motion objects detection is an important problem of computer vision and image coding. It is widely used in automaton navigation, intelligent video surveillance systems, traffic detection, medical image processing, video compression and transmission and so on. As a main large scale programmable specific integrated circuit, FPGA meets the requirements of high speed image processing, with the characteristics of simple algorithm realization, easy programming, good portability and inheritability. In this thesis, we develop a motion detection system based on FPGA. It is flexible for later changing of algorithm, image size and collection and display equipments.
By analyzing some of the main algorithm of motion detection, especially background subtraction, a new motion detection algorithm called improved W4 is presented. The novel algorithm has the advantages of background subtraction and overcomes the shortages of W4 algorithm that can not efficiently detect moving object in frequent varying environmental weather condition.
We use MATLAB to simulate the improved W4 algorithm, and then port it to the FPGA. We design image collection, motion detection and display modules to consummate the motion detection system. In order to accelerate it to be a real-time system, a fast algorithm which uses block instead of pixel in improved W4 algorithm is presented.
引文
[1] A.Azarbayejani, C.Wren, and A.Pentland.Real-time 3-d tracking of the human body.IEEE Transactions on Pattern Analysis and Machine Intelligence.Vol.19,No.7,pp 780-785,1997.
[2] Christian Schlegel,Heiko Jaberg,and Matthias Schuster.Vision Based Person Tracking with a Mobile Robot.British Machine Vision Conference.pp.419-427,1997.
[3] 王铨,艾海州,何克忠.基于差分图像的多运动目标的检测和跟踪.中国图像图形学报.1999,4(6):470-474
[4] l·KenncthR.Castlmena 著.朱志刚等译.数字图像处理.北京:电子工业出版社,2002 年 2 月第一版
[5] 刘晓东,苏光大,周全,田超.一种可视化智能户外监控系统.中国图像图形学报.2000,5(12):1024-1029
[6] I. smail Haritaoglu, D. Harvood, and L. S. Davis, “A fast background scene modeling and maintenance for outdoor surveillance,” IEEE International Conference on Pattern Recognition, vol.4, pp179-183, 2000
[7] A. Elgammal, D.Harwood and L.Davis, “Non-parametric model for background subtraction,” In IEEE Frame rate workshop,1999
[8] T.Horprasert, D.Harwood, and L.S.Davis, “A Statistical Approach for Real-time Robust Background Subtraction and Shadow Detection,” Proc. IEEE ICCV’99 FRAME-RATE Workshop, Kerkyra, Greece, 1999
[9] R.Cucchiara, C.Grana, and A.Prati, “Detection moving object, ghosts, and shadows in video streams,” IEEE transaction on pattern analysis and machine intelligence, vol.25, October. 2003
[10] J.C.S.J.Jr., C.R.Jung and S.R.Musse, “A background subtraction model adapted to illumination changes,” IEEE International Conference on image processing, pp.1817-1820, 2006
[11] 褚振勇,翁木云.FPGA 设计及应用.西安电子科技大学出版社,2002.6
[12] 宇野由麻子,大石基之.FPGA 产品市场现状与展望.电子设计应用,2004,60(12):63~71
[13] Y.Xiong, “Monent and hypergeometric filters for high precision computation of focus, stereo and optical flow,” International Journal of computer vision, pp.25-59, 1997
[14] Ming Ye, “Robust visual motion analysis: piecewise-smooth optical flow and motion-based detection and tracking,” University of Washington, 2002
[15] Mae.Y, Y.Shirai et al., “Object tracking in cluttered background based on optical flow and edges,” Proc the 13th International Conference on Pattern Recognition, pp.196-200, 1996
[16] Mae,Y, et al., “Optical Flow Based Real-time object tracking by active vision system,” Proc. 2nd Japan-France Congress on Mechatronics, pp.545-548, 1994
[17] A.J.Lipton, Fujiyoshi et al., “Moving target classification and tracking from real-time video,” Proc. IEEE Workshop on Applications of Computer Vision, pp.8-14, 1998
[18] A.Neri, S.Colonnese, G.Russ, et al., “Automatic moving object and background separation,” Signal Processin, pp.219-232 , 1998
[19] R.Mech, M.Wollborn, “A noise robust method for segmentation of moving objects in video sequences,” Proc International Conference on Acoustics speech and Signal Processing, pp.41-45, 1997
[20] F.Moscheni, S.Bhattach, et al., “Spatial temporal segmentation based on region merging,” IEEE Transaction on pattern analysis and machine intelliqence, pp.897-915,1998
[21] P.L.Rosin, T.Ellis, “Image difference threshold strategies and shadow detection,” Proc the 1995 British conference on Machine vision, pp.347-356, 1995
[22] S.McKenna , et al., “Tracking groups of people,” Computer Vision and Image Understanding, pp.42-56, 2000
[23] K.P.Karmann, A.Brandt, “Moving object recognition using an adaptive background memory,” Time-Varying Image Processing and Moving Object Recognition, 2, Elsevier, Amsterdam, The Netherlands, Vol.2, p.289-296, 1990
[24] M.Kilger, “A shadow handler in a video2based real-time traffic monitoring system,” Proc IEEE Workshop on Applications of Computer Vision, Palm Springs, CA, pp.1060-1066, 1992
[25] M.Kass, A.Witkin, D.Terzopoulous, “Snakes: active contour models,” International Journal of Computer Vision, pp.321-331, 1987
[26] S.Mallat, W.L.Hwang, “Singularity detection and processing with wavelets,” IEEE Transactions on IT, pp.617-643, 1992
[27] C.E.Daniell, D.J.Kemsley, W.P.Lincoln, et al., “Artificial neural networks for automatic target recognition,” Optical Engineering, pp.2521-2530, 1992
[28] I.Haritaoglu, and D.Harvood, “ w 4:Real-Time surveillance of people and their activities,” IEEE Ttransactions on Pattern Analysis and Machine Intelligence, vol.22, pp.804-830, 2000
[29] J.C.S.Jacques, C.R.Jung, and S.R.Musse, “Background subtraction and shadow detection in grayscale video sequences,” in Proceedings of SIBGRAPI, Natal, Brazil, pp. 189-196, IEEE Press, October. 2005
[30] A.M.Elgammal, R.Duraiswami, D.Harwood, and L.S.Davis, “Background and foreground modeling using nonparametric kernel density estimation for visual surveillance,” Proceeedings of the IEEE, vol. 90, no. 7, pp.1151-1163, 2002.
[31] W.E.L.Grimson, C.Stauffer, and R.Romano, “Using adaptive tracking to classify and monitor activities in a site,” in CVPR, 1998
[32] W.E.L.Grimson and C.Stauffer, “Adaptive background mixture models for real-time tracking,” in CVPR, 1999
[33] C.Stauffer and W.E. L.Grimson, “Learning patterns of activity using real-time tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no.8, pp. 747–757, Aug. 2000
[34] A. Elgammal, D.Harwood and L.Davis, “Non-parametric model for background subtraction,” In IEEE Frame rate workshop,1999
[35] A.Cavallaro and T.Ebrahimi, “Video object extraction based on adaptive background and statistical change detection,” in Proc.SPIE Visual Communications and Image Processing, pp. 465–475, Jan. 2001
[36] S.Huwer and H.Niemann, “Adaptive change detection for real-time surveillance applications,” in Proc. Visual Surveillance, 2000
[37] T.Kanade, R.Collins, A.Lipton, P.Burt, and L.Wixson, “Advances in cooperative multi-sensor video surveillance,” in Proc. DARPA Image Understanding Workshop, vol. 1, pp. 3–24, Nov. 1998
[38] N.Friedman and S.Russell, “Image segmentation in video sequences: A probabilistic approach,” in Uncertainty in Artificial Intelligence, 1997
[39] 吴喜之、王兆军. 非参数统计方法. 北京:高等教育出版社. 1996
[40] M.Kass, A.Witkin, D.Terzopoulous, “Snakes: active contour models,” International Journal of Computer Vision, pp.321-331, 1987
[41] S.Mallat, W.L.Hwang, “Singularity detection and processing with wavelets,” IEEE Transactions on IT, pp.617-643, 1992
[42] C.E.Daniell, D.J.Kemsley, W.P.Lincoln, et al., “Artificial neural networks for automatic target recognition,” Optical Engineering, pp.2521-2530, 1992
[43] I. smail Haritaoglu, D. Harvood, and L. S. Davis, “A fast background scene modeling and maintenance for outdoor surveillance,” IEEE International Conference on Pattern Recognition, vol.4, pp179-183, 2000
[44] A. Elgammal, D.Harwood and L.Davis, “Non-parametric model for background subtraction,” In IEEE Frame rate workshop,1999
[45] T.Horprasert, D.Harwood, and L.S.Davis, “A Statistical Approach for Real-time Robust Background Subtraction and Shadow Detection,” Proc. IEEE ICCV’99 FRAME-RATE Workshop, Kerkyra, Greece, 1999
[46] R.Cucchiara, C.Grana, and A.Prati, “Detection moving object, ghosts, and shadows in video streams,” IEEE transaction on pattern analysis and machine intelligence, vol.25, October. 2003
[47] J.C.S.J.Jr., C.R.Jung and S.R.Musse, “A background subtraction model adapted to illumination changes,” IEEE International Conference on image processing, pp.1817-1820, 2006
[48] M.Bramberger, J.Brunner, B.Rinner, and H.Schwabach, “Real-time video analysis on an embedded smart camera for traffic surveillance,” Real-time and Embedded Technology and Applications Symposium, pp. 174-181, 2004
[49] A.L.P.Douglas et al., “An efficient DSP implementation of real-time stationary vehicle detection by smart camera at outdoor conditions,” IEEE International Conference on image processing, pp.3275-3276, 2006
[2] Christian Schlegel,Heiko Jaberg,and Matthias Schuster.Vision Based Person Tracking with a Mobile Robot.British Machine Vision Conference.pp.419-427,1997.
[3] 王铨,艾海州,何克忠.基于差分图像的多运动目标的检测和跟踪.中国图像图形学报.1999,4(6):470-474
[4] l·KenncthR.Castlmena 著.朱志刚等译.数字图像处理.北京:电子工业出版社,2002 年 2 月第一版
[5] 刘晓东,苏光大,周全,田超.一种可视化智能户外监控系统.中国图像图形学报.2000,5(12):1024-1029
[6] I. smail Haritaoglu, D. Harvood, and L. S. Davis, “A fast background scene modeling and maintenance for outdoor surveillance,” IEEE International Conference on Pattern Recognition, vol.4, pp179-183, 2000
[7] A. Elgammal, D.Harwood and L.Davis, “Non-parametric model for background subtraction,” In IEEE Frame rate workshop,1999
[8] T.Horprasert, D.Harwood, and L.S.Davis, “A Statistical Approach for Real-time Robust Background Subtraction and Shadow Detection,” Proc. IEEE ICCV’99 FRAME-RATE Workshop, Kerkyra, Greece, 1999
[9] R.Cucchiara, C.Grana, and A.Prati, “Detection moving object, ghosts, and shadows in video streams,” IEEE transaction on pattern analysis and machine intelligence, vol.25, October. 2003
[10] J.C.S.J.Jr., C.R.Jung and S.R.Musse, “A background subtraction model adapted to illumination changes,” IEEE International Conference on image processing, pp.1817-1820, 2006
[11] 褚振勇,翁木云.FPGA 设计及应用.西安电子科技大学出版社,2002.6
[12] 宇野由麻子,大石基之.FPGA 产品市场现状与展望.电子设计应用,2004,60(12):63~71
[13] Y.Xiong, “Monent and hypergeometric filters for high precision computation of focus, stereo and optical flow,” International Journal of computer vision, pp.25-59, 1997
[14] Ming Ye, “Robust visual motion analysis: piecewise-smooth optical flow and motion-based detection and tracking,” University of Washington, 2002
[15] Mae.Y, Y.Shirai et al., “Object tracking in cluttered background based on optical flow and edges,” Proc the 13th International Conference on Pattern Recognition, pp.196-200, 1996
[16] Mae,Y, et al., “Optical Flow Based Real-time object tracking by active vision system,” Proc. 2nd Japan-France Congress on Mechatronics, pp.545-548, 1994
[17] A.J.Lipton, Fujiyoshi et al., “Moving target classification and tracking from real-time video,” Proc. IEEE Workshop on Applications of Computer Vision, pp.8-14, 1998
[18] A.Neri, S.Colonnese, G.Russ, et al., “Automatic moving object and background separation,” Signal Processin, pp.219-232 , 1998
[19] R.Mech, M.Wollborn, “A noise robust method for segmentation of moving objects in video sequences,” Proc International Conference on Acoustics speech and Signal Processing, pp.41-45, 1997
[20] F.Moscheni, S.Bhattach, et al., “Spatial temporal segmentation based on region merging,” IEEE Transaction on pattern analysis and machine intelliqence, pp.897-915,1998
[21] P.L.Rosin, T.Ellis, “Image difference threshold strategies and shadow detection,” Proc the 1995 British conference on Machine vision, pp.347-356, 1995
[22] S.McKenna , et al., “Tracking groups of people,” Computer Vision and Image Understanding, pp.42-56, 2000
[23] K.P.Karmann, A.Brandt, “Moving object recognition using an adaptive background memory,” Time-Varying Image Processing and Moving Object Recognition, 2, Elsevier, Amsterdam, The Netherlands, Vol.2, p.289-296, 1990
[24] M.Kilger, “A shadow handler in a video2based real-time traffic monitoring system,” Proc IEEE Workshop on Applications of Computer Vision, Palm Springs, CA, pp.1060-1066, 1992
[25] M.Kass, A.Witkin, D.Terzopoulous, “Snakes: active contour models,” International Journal of Computer Vision, pp.321-331, 1987
[26] S.Mallat, W.L.Hwang, “Singularity detection and processing with wavelets,” IEEE Transactions on IT, pp.617-643, 1992
[27] C.E.Daniell, D.J.Kemsley, W.P.Lincoln, et al., “Artificial neural networks for automatic target recognition,” Optical Engineering, pp.2521-2530, 1992
[28] I.Haritaoglu, and D.Harvood, “ w 4:Real-Time surveillance of people and their activities,” IEEE Ttransactions on Pattern Analysis and Machine Intelligence, vol.22, pp.804-830, 2000
[29] J.C.S.Jacques, C.R.Jung, and S.R.Musse, “Background subtraction and shadow detection in grayscale video sequences,” in Proceedings of SIBGRAPI, Natal, Brazil, pp. 189-196, IEEE Press, October. 2005
[30] A.M.Elgammal, R.Duraiswami, D.Harwood, and L.S.Davis, “Background and foreground modeling using nonparametric kernel density estimation for visual surveillance,” Proceeedings of the IEEE, vol. 90, no. 7, pp.1151-1163, 2002.
[31] W.E.L.Grimson, C.Stauffer, and R.Romano, “Using adaptive tracking to classify and monitor activities in a site,” in CVPR, 1998
[32] W.E.L.Grimson and C.Stauffer, “Adaptive background mixture models for real-time tracking,” in CVPR, 1999
[33] C.Stauffer and W.E. L.Grimson, “Learning patterns of activity using real-time tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no.8, pp. 747–757, Aug. 2000
[34] A. Elgammal, D.Harwood and L.Davis, “Non-parametric model for background subtraction,” In IEEE Frame rate workshop,1999
[35] A.Cavallaro and T.Ebrahimi, “Video object extraction based on adaptive background and statistical change detection,” in Proc.SPIE Visual Communications and Image Processing, pp. 465–475, Jan. 2001
[36] S.Huwer and H.Niemann, “Adaptive change detection for real-time surveillance applications,” in Proc. Visual Surveillance, 2000
[37] T.Kanade, R.Collins, A.Lipton, P.Burt, and L.Wixson, “Advances in cooperative multi-sensor video surveillance,” in Proc. DARPA Image Understanding Workshop, vol. 1, pp. 3–24, Nov. 1998
[38] N.Friedman and S.Russell, “Image segmentation in video sequences: A probabilistic approach,” in Uncertainty in Artificial Intelligence, 1997
[39] 吴喜之、王兆军. 非参数统计方法. 北京:高等教育出版社. 1996
[40] M.Kass, A.Witkin, D.Terzopoulous, “Snakes: active contour models,” International Journal of Computer Vision, pp.321-331, 1987
[41] S.Mallat, W.L.Hwang, “Singularity detection and processing with wavelets,” IEEE Transactions on IT, pp.617-643, 1992
[42] C.E.Daniell, D.J.Kemsley, W.P.Lincoln, et al., “Artificial neural networks for automatic target recognition,” Optical Engineering, pp.2521-2530, 1992
[43] I. smail Haritaoglu, D. Harvood, and L. S. Davis, “A fast background scene modeling and maintenance for outdoor surveillance,” IEEE International Conference on Pattern Recognition, vol.4, pp179-183, 2000
[44] A. Elgammal, D.Harwood and L.Davis, “Non-parametric model for background subtraction,” In IEEE Frame rate workshop,1999
[45] T.Horprasert, D.Harwood, and L.S.Davis, “A Statistical Approach for Real-time Robust Background Subtraction and Shadow Detection,” Proc. IEEE ICCV’99 FRAME-RATE Workshop, Kerkyra, Greece, 1999
[46] R.Cucchiara, C.Grana, and A.Prati, “Detection moving object, ghosts, and shadows in video streams,” IEEE transaction on pattern analysis and machine intelligence, vol.25, October. 2003
[47] J.C.S.J.Jr., C.R.Jung and S.R.Musse, “A background subtraction model adapted to illumination changes,” IEEE International Conference on image processing, pp.1817-1820, 2006
[48] M.Bramberger, J.Brunner, B.Rinner, and H.Schwabach, “Real-time video analysis on an embedded smart camera for traffic surveillance,” Real-time and Embedded Technology and Applications Symposium, pp. 174-181, 2004
[49] A.L.P.Douglas et al., “An efficient DSP implementation of real-time stationary vehicle detection by smart camera at outdoor conditions,” IEEE International Conference on image processing, pp.3275-3276, 2006