基于FPGA远程网络监视系统
|
摘要
近年来随着多媒体技术的迅速发展和计算机性能的不断提高,数字视频监视系统已逐渐取代传统的模拟式监视系统而广泛应用于银行、小区、宾馆等公共场所,并在公共安全领域发挥着重要的作用。在未来的发展中,智能化监视系统将有广阔的应用前景。
本论文研究开发了一种基于运动目标识别技术的数字视频远传网络监视系统。本系统采用QuartusⅡ和NiosⅡ嵌入式开发软件,将图像处理、网络传输、SOPC(System on a Programmable Chip可编程片上系统)技术紧密连接在一起,并采用目标提取算法减少视频大数据量传输,提高视频传输的实时性和显示质量,为远程视频监视提供了一种新的解决方法。本设计通过摄像头采集图像、网络传输背景及运动目标,对现场有无人员入侵进行判断,实现无人值守的视频监视。论文探讨了一种基于SOPC技术的视频监视系统的设计方案——CPU (Central Processing Unit中央处理器)结合用户自定义逻辑设计的方案。此方案的设计思想是以下载到FPGA(Field Programmable Gate Array片上可编程逻辑门阵列)的NiosⅡ5.0嵌入式软核CPU为系统控制模块,实现数据流的接收、处理、控制和UDP (User Datagram Protocol用户数据报协议)协议网络传输,并运用FPGA逻辑单元实现视频图像采集和处理。本方案中视频采集和处理程序采用Verilog语言来进行设计,在很大程度上提高了系统速度。在软硬件协同设计方面,采用摄像头自定义组件模块挂接Avalon总线的方法,通过DMA (Direct Memory Access直接存储器访问)将硬件视频数据传输至SDRAM (Synchronous Dynamic RAM同步动态存储器)。在图像处理实现方面,采用相邻两帧图片进行差分的算法实现运动目标的提取,在网络传输视频数据时对背景和运动目标分别进行传输,降低了数据传输量,提高了传输速度。
本监视系统核心技术是视频采集、转换、运动目标检测、提取及通信技术。本文主要研究内容包括视频采集、颜色模式转换、分辨率转换、图像数据的传输和运动目标检测等问题。在硬件系统集成过程中,系统采用专业的130万像素摄像头,并配以自定义组件模块实现其视频采集、预处理功能,并在SOPC Builder软件中配置硬件集成系统。在软件编程中采用C语言,编写了运动目标提取、网络有线传输等相关函数,并在此基础上构建软件系统。最后本文通过时序仿真判断硬件连接的正确性和用EtherPeek软件捕获UDP协议网络数据包来判断网络连接、视频传输和C语言编写函数的正确性。
In recent years, with the rapid development of multimedia technology and the increasing enhancement of computer performance, the digital video surveillance system has gradually substituted for traditional analog video surveillance system, which is widely applied in public places, such as banks, intelligent communities and hotels. It plays an important role in the public security field. Therefore, the intelligent video surveillance system will own a bright application prospect in the future.
A digital video remote network surveillance system based on the concept of moving object recognition was designed in this paper. This system adopted the embedded softwares QuartusII and NiosⅡ, and integrated video processing, network transmitting and SOPC embedded technology. Meanwhile, it applied the algorithm of object distillation to reduce a great number of transmitting data and improve the performance of real time and display quality, which presented a new solution available to the system. By video capturing, background transmitting and moving objects transmitting via network, the design can judge whether invading objects exist and realize surveillance system without attendant. A video surveillance system design plan based on SOPC technology,——the plan of CPU integrated with user's self-defining logic, was discussed in the thesis. CPU NiosⅡ5.0 embedded IP core, which has been downloaded to FPGA, acts as the control module of the system. It can receive、process、control data stream, and network transmitting based on UDP protocol. FPGA's logic units are used to implement the video image manipulation and processing. To a large extent, this plan has improved the speed, of the system, because the system's hardware program was written in Verilog HDL language, in order to capture and process video data. In the way of collaborating design of software and hardware, the system adopt the solutions that video user's self-defining module linked with Avalon bus and that DMA module transferred hardware video data to SDRAM memory. In the way of realizing image manipulation, the system realized moving object recognition through adopting the difference formula of two neighboring pictures. When the system transmits video data through network, the background pictures and moving object pictures are transmitted separately. So the solution reduces the great number of transmitting data and improves transmitting rate and display quality.
The key technology of video surveillance system is video capture, video transmitting, moving-target detection, recognition and communication. The paper has realized a type of remote network surveillance system based on FPGA, and has solved several problems, including video capture, color mode transition, resolution change, video data transition and moving-target detection. In the hardware system integration part, this system adopted specialized mega-pixel camera、configured user logic component module、realized the function of gathering、processing video, and built hardware integrated system based on SOPC, as well as compiled NiosⅡC/C++ software system through the function of moving-target recognition and network communication. Finial, the paper confirmed the correctness of hardware communication through timing simulator and the correctness of network communication and C program.
本论文研究开发了一种基于运动目标识别技术的数字视频远传网络监视系统。本系统采用QuartusⅡ和NiosⅡ嵌入式开发软件,将图像处理、网络传输、SOPC(System on a Programmable Chip可编程片上系统)技术紧密连接在一起,并采用目标提取算法减少视频大数据量传输,提高视频传输的实时性和显示质量,为远程视频监视提供了一种新的解决方法。本设计通过摄像头采集图像、网络传输背景及运动目标,对现场有无人员入侵进行判断,实现无人值守的视频监视。论文探讨了一种基于SOPC技术的视频监视系统的设计方案——CPU (Central Processing Unit中央处理器)结合用户自定义逻辑设计的方案。此方案的设计思想是以下载到FPGA(Field Programmable Gate Array片上可编程逻辑门阵列)的NiosⅡ5.0嵌入式软核CPU为系统控制模块,实现数据流的接收、处理、控制和UDP (User Datagram Protocol用户数据报协议)协议网络传输,并运用FPGA逻辑单元实现视频图像采集和处理。本方案中视频采集和处理程序采用Verilog语言来进行设计,在很大程度上提高了系统速度。在软硬件协同设计方面,采用摄像头自定义组件模块挂接Avalon总线的方法,通过DMA (Direct Memory Access直接存储器访问)将硬件视频数据传输至SDRAM (Synchronous Dynamic RAM同步动态存储器)。在图像处理实现方面,采用相邻两帧图片进行差分的算法实现运动目标的提取,在网络传输视频数据时对背景和运动目标分别进行传输,降低了数据传输量,提高了传输速度。
本监视系统核心技术是视频采集、转换、运动目标检测、提取及通信技术。本文主要研究内容包括视频采集、颜色模式转换、分辨率转换、图像数据的传输和运动目标检测等问题。在硬件系统集成过程中,系统采用专业的130万像素摄像头,并配以自定义组件模块实现其视频采集、预处理功能,并在SOPC Builder软件中配置硬件集成系统。在软件编程中采用C语言,编写了运动目标提取、网络有线传输等相关函数,并在此基础上构建软件系统。最后本文通过时序仿真判断硬件连接的正确性和用EtherPeek软件捕获UDP协议网络数据包来判断网络连接、视频传输和C语言编写函数的正确性。
In recent years, with the rapid development of multimedia technology and the increasing enhancement of computer performance, the digital video surveillance system has gradually substituted for traditional analog video surveillance system, which is widely applied in public places, such as banks, intelligent communities and hotels. It plays an important role in the public security field. Therefore, the intelligent video surveillance system will own a bright application prospect in the future.
A digital video remote network surveillance system based on the concept of moving object recognition was designed in this paper. This system adopted the embedded softwares QuartusII and NiosⅡ, and integrated video processing, network transmitting and SOPC embedded technology. Meanwhile, it applied the algorithm of object distillation to reduce a great number of transmitting data and improve the performance of real time and display quality, which presented a new solution available to the system. By video capturing, background transmitting and moving objects transmitting via network, the design can judge whether invading objects exist and realize surveillance system without attendant. A video surveillance system design plan based on SOPC technology,——the plan of CPU integrated with user's self-defining logic, was discussed in the thesis. CPU NiosⅡ5.0 embedded IP core, which has been downloaded to FPGA, acts as the control module of the system. It can receive、process、control data stream, and network transmitting based on UDP protocol. FPGA's logic units are used to implement the video image manipulation and processing. To a large extent, this plan has improved the speed, of the system, because the system's hardware program was written in Verilog HDL language, in order to capture and process video data. In the way of collaborating design of software and hardware, the system adopt the solutions that video user's self-defining module linked with Avalon bus and that DMA module transferred hardware video data to SDRAM memory. In the way of realizing image manipulation, the system realized moving object recognition through adopting the difference formula of two neighboring pictures. When the system transmits video data through network, the background pictures and moving object pictures are transmitted separately. So the solution reduces the great number of transmitting data and improves transmitting rate and display quality.
The key technology of video surveillance system is video capture, video transmitting, moving-target detection, recognition and communication. The paper has realized a type of remote network surveillance system based on FPGA, and has solved several problems, including video capture, color mode transition, resolution change, video data transition and moving-target detection. In the hardware system integration part, this system adopted specialized mega-pixel camera、configured user logic component module、realized the function of gathering、processing video, and built hardware integrated system based on SOPC, as well as compiled NiosⅡC/C++ software system through the function of moving-target recognition and network communication. Finial, the paper confirmed the correctness of hardware communication through timing simulator and the correctness of network communication and C program.
引文
[1]何立民,嵌入式系统的定义与发展历史.北京：北京航空航天大学,2003
[2]李华,杨军山,电视系统中的运动检测和运动估值,电视技术,1995,6(2)：2-6
[3]王积分,张新荣,计算机图像识别.北京：中国铁道出版社,1988
[4]张大波,嵌入式系统原理设计与应用.北京：机械工业出版社,2002
[5]潘松,黄继业,SOPC技术实用教程.北京：清华大学出版社,2005
[6]汪国强,SOPC技术与应用.北京：机械工业出版社,2006
[7]彭澄廉,挑战SOC——基于Nios的SOPC设计与实践.北京：清华大学出版社,2004
[8]Michael D.Ciletti. Advanced Digital Design with the Verilog HDL.beijing:Publishing House of Electronics Industry,2005
[9]Spartan-II 2.5V FPGA Family:Complete Data Sheet Product Specification. Philips Semiconductors.2004 Auf 2
[10]Altera Corporation, NiosⅡ Embedded Processor Design Contest Outstanding Designs 2006: www.Altera.com.cn
[11]Altera Corporation, NiosII Processor Reference Handbook:www.Altera.com.cn
[12]易克初,田斌,李刚强,FPGA设计中关键问题的研究,电子技术应用,2003,6(4),55-58
[13]夏宇闻,Verilog数字系统设计教程.北京：北京航空航天大学出版社,2003
[14]Jan M.Rabacy, Anantha Chandrakasan, Borivoje Nikolic,数字集成电路设计透视.北京：清华大学出版社,2004
[15]薛宏熙,胡秀珠,计算机组成与设计.北京：清华大学出版社,2007
[16]徐光辉,程东旭,基于FPGA的嵌入式开发与应用.北京：电子工业出版社,2006
[17]黄智伟,FPGA系统设计与实践.北京：电子工业出版社,2005
[18]易国华,灰度图像序列中运动目标提取方法的研究,鄂州大学学报,2003,10(4)：25-27
[19]刘永,戴礼容,宋彦,一种静态背景下的运动目标提取算法,中国图像学报,2003,7(5)：760-163
[20]周新伦,柳健,刘华志,数字图像处理.北京：国防工业出版社,1986
[21]李金宗,李宁宁,图像积累与自适应门限检测技术的研究,电子学报,1997,25(13)：55-59
[22]阮秋琦,数字图像处理学.北京:电子工业出版社,2001
[23]李彬,刘冀伟,韩鸿哲等.复杂背景下人体骨架的提取[J],微计算机信息,2004,20(7)：43-44
[24]李春明,李玉山,张大朴等,运动人物的检测、跟踪与识别综述[DB/OL]. http://www.paper.edu.cn,2005.04-18.
[25]Neff A, Colonnese S, Russo G, Talone P. Automatic moving object and background separation[J]. Signal Processing,1998,66(2):219-232
[26]Me chR, WolbomM. A noise robust method for 2D shape of moving object in video sequences commandeering annoying camera [J]. Signal Processing,1998,66(2):203-205
[27]A Shio, J Sklansky. Segmentation of people in motion. Proc. IEEE Workshop on Visual Motion [C].1991.325-332.
[28]张建荣,姜昱明,实时跟踪系统中运动人体图像分割[J],计算机仿真,2004,21(6)：54-56
[29]唐常青,吕宏伯,数学形态学方法及其应用.北京：科学出版社,1999.
[30]龚炜,石青云,程民德,数字空间中的数学形态学——理论及应用.北京：科学出版社,1997
[31]乔晓丹,张鹏,一个基于Linux操作系统的嵌入式网关的实现[J],微计算机信息,2005.21(15),34-37
[32]朱刚,Linux网络编程,北京：科学出版社,2000.
[33]戴青云,余英林,数学形态学在图像处理中的应用进展[J],控制理论应用,2001,8(4)：478-482
[34]Hochong Park, Roland T Chin. Decomposition of arbitrarily shaped morphological structuring elements [J]. IEEE Trans on PAMI,1991,7(1):2-15.
[35]龚炜,数学形态学中结构元素的分解[J],高校应用数学学报,1998,7(3)：340-349
[36]R. Brunelli, O. Mich, C.M. Modena. A survey on the automatic index of video data, J. Visual Common. Image Representation,1999,10(7):78-112
[37]R. Koenen, F. Pereira, L. Chiariglione. MPEG-4:context and objectives, Signal Process, Image Common.1997,9 (4):295-304
[38]J. Martinez, R. Koenen, F. Pereira. MPEG-7:the generic multimedia content description standard, part 1, multimedia, IEEE 2002,9 (6):78-87
[39]D.S. Zhang, G. Lu. Segmentation of moving objects in image sequence:a review, Circuits Syst. Signal Process.2001,20(2):143-183
[40]P.L. Rosin. Threshold for change detection, in:Proceedings of the Sixth International Conference on Computer Vision. IEEE Computer Society, Silver Spring,MD,1998,274p.
[41]P. Giaccone, G. Jones. Segmentation of global motion using temporal probabilistic classification, in:British Machine Vision Conference,1998, pp.619-628.
[42]H. Sawhney, S. Ayer. Compact representations of videos through dominant and multiple motion estimation, IEEE Trans. Pattern Analy.Mach. Intell.1996,18 (8):814-830.
[43]M. Irani, P. Anandan, J. Bergen, R. Kumar, S. Hsu. Efficient representations of video sequences and their applications, Signal Process. Image Common.1996,8(4):327-351.
[44]S. Park, J. Aggarwal. Segmentation and tracking of interacting human body parts under occlusion and shadowing, in:IEEE Workshop on Motion and Video Computing,2002, pp. 105-111.
[45]I. Cohen, G Medioni. Detecting and tracking moving objects in video surveillance, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,1999, pp.Ⅱ: 319-325.
[2]李华,杨军山,电视系统中的运动检测和运动估值,电视技术,1995,6(2)：2-6
[3]王积分,张新荣,计算机图像识别.北京：中国铁道出版社,1988
[4]张大波,嵌入式系统原理设计与应用.北京：机械工业出版社,2002
[5]潘松,黄继业,SOPC技术实用教程.北京：清华大学出版社,2005
[6]汪国强,SOPC技术与应用.北京：机械工业出版社,2006
[7]彭澄廉,挑战SOC——基于Nios的SOPC设计与实践.北京：清华大学出版社,2004
[8]Michael D.Ciletti. Advanced Digital Design with the Verilog HDL.beijing:Publishing House of Electronics Industry,2005
[9]Spartan-II 2.5V FPGA Family:Complete Data Sheet Product Specification. Philips Semiconductors.2004 Auf 2
[10]Altera Corporation, NiosⅡ Embedded Processor Design Contest Outstanding Designs 2006: www.Altera.com.cn
[11]Altera Corporation, NiosII Processor Reference Handbook:www.Altera.com.cn
[12]易克初,田斌,李刚强,FPGA设计中关键问题的研究,电子技术应用,2003,6(4),55-58
[13]夏宇闻,Verilog数字系统设计教程.北京：北京航空航天大学出版社,2003
[14]Jan M.Rabacy, Anantha Chandrakasan, Borivoje Nikolic,数字集成电路设计透视.北京：清华大学出版社,2004
[15]薛宏熙,胡秀珠,计算机组成与设计.北京：清华大学出版社,2007
[16]徐光辉,程东旭,基于FPGA的嵌入式开发与应用.北京：电子工业出版社,2006
[17]黄智伟,FPGA系统设计与实践.北京：电子工业出版社,2005
[18]易国华,灰度图像序列中运动目标提取方法的研究,鄂州大学学报,2003,10(4)：25-27
[19]刘永,戴礼容,宋彦,一种静态背景下的运动目标提取算法,中国图像学报,2003,7(5)：760-163
[20]周新伦,柳健,刘华志,数字图像处理.北京：国防工业出版社,1986
[21]李金宗,李宁宁,图像积累与自适应门限检测技术的研究,电子学报,1997,25(13)：55-59
[22]阮秋琦,数字图像处理学.北京:电子工业出版社,2001
[23]李彬,刘冀伟,韩鸿哲等.复杂背景下人体骨架的提取[J],微计算机信息,2004,20(7)：43-44
[24]李春明,李玉山,张大朴等,运动人物的检测、跟踪与识别综述[DB/OL]. http://www.paper.edu.cn,2005.04-18.
[25]Neff A, Colonnese S, Russo G, Talone P. Automatic moving object and background separation[J]. Signal Processing,1998,66(2):219-232
[26]Me chR, WolbomM. A noise robust method for 2D shape of moving object in video sequences commandeering annoying camera [J]. Signal Processing,1998,66(2):203-205
[27]A Shio, J Sklansky. Segmentation of people in motion. Proc. IEEE Workshop on Visual Motion [C].1991.325-332.
[28]张建荣,姜昱明,实时跟踪系统中运动人体图像分割[J],计算机仿真,2004,21(6)：54-56
[29]唐常青,吕宏伯,数学形态学方法及其应用.北京：科学出版社,1999.
[30]龚炜,石青云,程民德,数字空间中的数学形态学——理论及应用.北京：科学出版社,1997
[31]乔晓丹,张鹏,一个基于Linux操作系统的嵌入式网关的实现[J],微计算机信息,2005.21(15),34-37
[32]朱刚,Linux网络编程,北京：科学出版社,2000.
[33]戴青云,余英林,数学形态学在图像处理中的应用进展[J],控制理论应用,2001,8(4)：478-482
[34]Hochong Park, Roland T Chin. Decomposition of arbitrarily shaped morphological structuring elements [J]. IEEE Trans on PAMI,1991,7(1):2-15.
[35]龚炜,数学形态学中结构元素的分解[J],高校应用数学学报,1998,7(3)：340-349
[36]R. Brunelli, O. Mich, C.M. Modena. A survey on the automatic index of video data, J. Visual Common. Image Representation,1999,10(7):78-112
[37]R. Koenen, F. Pereira, L. Chiariglione. MPEG-4:context and objectives, Signal Process, Image Common.1997,9 (4):295-304
[38]J. Martinez, R. Koenen, F. Pereira. MPEG-7:the generic multimedia content description standard, part 1, multimedia, IEEE 2002,9 (6):78-87
[39]D.S. Zhang, G. Lu. Segmentation of moving objects in image sequence:a review, Circuits Syst. Signal Process.2001,20(2):143-183
[40]P.L. Rosin. Threshold for change detection, in:Proceedings of the Sixth International Conference on Computer Vision. IEEE Computer Society, Silver Spring,MD,1998,274p.
[41]P. Giaccone, G. Jones. Segmentation of global motion using temporal probabilistic classification, in:British Machine Vision Conference,1998, pp.619-628.
[42]H. Sawhney, S. Ayer. Compact representations of videos through dominant and multiple motion estimation, IEEE Trans. Pattern Analy.Mach. Intell.1996,18 (8):814-830.
[43]M. Irani, P. Anandan, J. Bergen, R. Kumar, S. Hsu. Efficient representations of video sequences and their applications, Signal Process. Image Common.1996,8(4):327-351.
[44]S. Park, J. Aggarwal. Segmentation and tracking of interacting human body parts under occlusion and shadowing, in:IEEE Workshop on Motion and Video Computing,2002, pp. 105-111.
[45]I. Cohen, G Medioni. Detecting and tracking moving objects in video surveillance, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,1999, pp.Ⅱ: 319-325.