嵌入式运动物体自动跟踪视频监控器的设计实现
|
摘要
随着中国经济的快速发展和人民生活水平的提高,国内外的一些恐怖事件和工厂矿井等突发事件促使了人民的安防和自我保护意识的提高。同时,近几年来计算机硬件和软件技术遵循着摩尔定律高速发展,使得安防产业的科学化、信息化和智能化成为了可能。
本文设计了基于Linux操作系统、PXA270嵌入式微处理器硬件平台、采用高清IP摄像机和云台实现的嵌入式运动物体自动跟踪监控器系统。该设计能够驱动云台旋转来监控和跟踪运动目标,使得被跟踪的目标一直出现在监控画面中,直到目标丢失。
该论文主要从两个方面进行阐述：
1、算法的分析与设计：
这一部分主要完成了高效的MJPEG图像分离算法,在帧差法和背景差法的基础上提出了新的帧差法提取背景的算法。通过详细分析和研究了OpenCV库中提供的最小特征值的角点提取算法,采用定位在运动目标上的离散角点来描述运动目标,最后详细分析和研究了LK的金字塔算法,采用了OpenCV库提供的LK金字塔光流算法在当前帧中查找运动目标,计算出运动目标的中心在当前画面中的位置,然后驱动云台运动,使得目标物体逐渐向画面的中央进行逼近,直到目标运动物体丢失(反应在程序处理中是描述运动物体的所有离散角点的完全丢失),最后停止云台的运动,重新进行以上的背景提取和目标计算的过程循环。
2、程序的移植和优化：
由于设计最终的运行平台是在武耀博德公司的PXA270实验平台上,设计中采用了大量的复杂算法,必须采用PXA270的Wireless MMX技术来优化程序,使得处理器的性能发挥到最大。该实验平台给出的交叉编译器不支持Wireless MMX指令集,运行的Linux操作系统为Linux2.4内核,均不能发挥出处理器的最大性能。
该部分主要完成了两个内容：
1)、PXA270平台的搭建：
首先采用了GUN提供的cross-tool为PXA270定制并编译生成了一个支持WirelessMMX和EABI(The Embedded Application Binary Interface)的编译器,然后针对PXA270的硬件平台移植了Linux2.6.28内核,开启了操作系统的浮点模拟器、WMMX技术、FrameBuffer和网络端口的支持,最后采用BusyBox制作了自己的文件系统,使得运行在PXA270上的操作系统得到最大的优化。
2)、OpenCV库和程序的优化
Intel提出的Wireless MMX技术采用了SIMD指令集,可以同时对64位、最大8个单元的字节数据同时运算。在程序中没有加入WMMX指令集的时候,编译器是无法智能的为代码自动产生WMMX技术支持的汇编指令。程序中,背景提取、特征点的计算、LK光流算法均含有消耗CPU时间的大量矩阵浮点计算,在详细研究了Intel给出的WMMX指令集后,完成了对程序中使用到的OpenCV库函数的优化和程序本身的优化。
由于PXA270处理器本身并不支持浮点运算指令,采用的是Linux内核中提供的浮点模拟器,所以程序的运行效果并不能同使用PC平台下支持浮点运算的x86处理器的效果相比。在降低了程序内部处理的图像尺寸后,实现了在PXA270平台下采集640*480图像,以FPS在5帧左右的处理性能完成了该设计。
With the rapid development of China's economy and the improvement of people's life, terrorist attacks at home and abroad and the sudden mines incidents prompted the people's security and self-protection awareness. Meanwhile, in recent years,computer hardware and software technology, following the rapid development of Moore's Law, brings about the possibility for the scientificalization, informationization, intelligencilization of the security industry.
This article designed a objective tracking system which is based on the Linux operating system and PXA270 embedded microprocessor hardware platform, adopts high-definition IP cameras and cradle head. Through driving the rotation of the cradle head, this design can track and monitor moving objective in complicated conditions and keep objective in the controlling pictures until objective is lost.
The paper described this from two aspects:
1. Algorithm Analysis and Design:
This section mainly completes an efficient separation algorithm for MJPEG images and gives a new background extraction algorithm based on the frame difference algorithm. It adopts separation corner positioned on moving objective to describe moving objective through detailed analysis and study of minimum values of the corner detection algorithm provided by the OpenCV library. Finally the article analyses and studies the LK's pyramid algorithm in detail, adopts the LK pyramid sparse optical flow algorithm provided by OpenCV library to find the current frame moving target, calculates the location in the current screen of the moving target'center. Then drives the cradle head movement, and makes the target object gradually approximate to the central screen, until the goal of moving objects is lost, and finally stops the movement of the cradle head. The above process of background extraction and objectives calculation will be cycled.
2. Porting and Optimizing Existing Code
As the final execution platform is the PXA270 board developed by 51 Board company and this design takes use of a large number of complex algorithms, which entails the optimization of the code by Wireless MMX technology in order to maximize the performance of the processor. The cross compiler this company provided does not support Wireless MMX instruction set and the operating system is kernel Linux2.4, neither are able to play the best performance.
The main part of the completion of two elements
1)、Establishing PXA270 platform
First I adopt the cross-tool provided by GUN and generate a compiler which is specified for the PXA270 and support Wireless MMX and EABI (The Embedded Application Binary Interface), then I port Linux2.6.28 kernel to the PXA270 board, enable the support of operating system's floating-point simulator, WMMX technology, FrameBuffer and network. At last, I use the BusyBox to complete my file system, achieving the maximum optimization for the running system on the PXA270.
2)、Optimizing OpenCV libraries and code
Wireless WMMX technology proposed by Intel uses SIMD instruction set, it can computer at the same time 64-bit, up to 8 units of bytes data. Without WMMX instruction in code, the compiler is not smart to generate automatically assembly instructions supported by WMMX technology. In the code, background extraction, feature point calculation, LK's optical flow algorithm are consuming a large number of CPU time by matrix, floating-point calculations. I completed the optimization of the OpenCV library and my code after the detailed study of the WMMX Intel instruction set.
As the PXA270 processor does not support floating-point instructions, I uses floating-point emulator provided by the Linux kernel. The efficient of process executed in PXA270 can not compare with in x86 processor with support for floating-point instruction. With collecting 640*480 images in the PXA270 platform I complete the design at about 5 FPS in the processing performance after decrease of the image size in the internal processing procedures.
本文设计了基于Linux操作系统、PXA270嵌入式微处理器硬件平台、采用高清IP摄像机和云台实现的嵌入式运动物体自动跟踪监控器系统。该设计能够驱动云台旋转来监控和跟踪运动目标,使得被跟踪的目标一直出现在监控画面中,直到目标丢失。
该论文主要从两个方面进行阐述：
1、算法的分析与设计：
这一部分主要完成了高效的MJPEG图像分离算法,在帧差法和背景差法的基础上提出了新的帧差法提取背景的算法。通过详细分析和研究了OpenCV库中提供的最小特征值的角点提取算法,采用定位在运动目标上的离散角点来描述运动目标,最后详细分析和研究了LK的金字塔算法,采用了OpenCV库提供的LK金字塔光流算法在当前帧中查找运动目标,计算出运动目标的中心在当前画面中的位置,然后驱动云台运动,使得目标物体逐渐向画面的中央进行逼近,直到目标运动物体丢失(反应在程序处理中是描述运动物体的所有离散角点的完全丢失),最后停止云台的运动,重新进行以上的背景提取和目标计算的过程循环。
2、程序的移植和优化：
由于设计最终的运行平台是在武耀博德公司的PXA270实验平台上,设计中采用了大量的复杂算法,必须采用PXA270的Wireless MMX技术来优化程序,使得处理器的性能发挥到最大。该实验平台给出的交叉编译器不支持Wireless MMX指令集,运行的Linux操作系统为Linux2.4内核,均不能发挥出处理器的最大性能。
该部分主要完成了两个内容：
1)、PXA270平台的搭建：
首先采用了GUN提供的cross-tool为PXA270定制并编译生成了一个支持WirelessMMX和EABI(The Embedded Application Binary Interface)的编译器,然后针对PXA270的硬件平台移植了Linux2.6.28内核,开启了操作系统的浮点模拟器、WMMX技术、FrameBuffer和网络端口的支持,最后采用BusyBox制作了自己的文件系统,使得运行在PXA270上的操作系统得到最大的优化。
2)、OpenCV库和程序的优化
Intel提出的Wireless MMX技术采用了SIMD指令集,可以同时对64位、最大8个单元的字节数据同时运算。在程序中没有加入WMMX指令集的时候,编译器是无法智能的为代码自动产生WMMX技术支持的汇编指令。程序中,背景提取、特征点的计算、LK光流算法均含有消耗CPU时间的大量矩阵浮点计算,在详细研究了Intel给出的WMMX指令集后,完成了对程序中使用到的OpenCV库函数的优化和程序本身的优化。
由于PXA270处理器本身并不支持浮点运算指令,采用的是Linux内核中提供的浮点模拟器,所以程序的运行效果并不能同使用PC平台下支持浮点运算的x86处理器的效果相比。在降低了程序内部处理的图像尺寸后,实现了在PXA270平台下采集640*480图像,以FPS在5帧左右的处理性能完成了该设计。
With the rapid development of China's economy and the improvement of people's life, terrorist attacks at home and abroad and the sudden mines incidents prompted the people's security and self-protection awareness. Meanwhile, in recent years,computer hardware and software technology, following the rapid development of Moore's Law, brings about the possibility for the scientificalization, informationization, intelligencilization of the security industry.
This article designed a objective tracking system which is based on the Linux operating system and PXA270 embedded microprocessor hardware platform, adopts high-definition IP cameras and cradle head. Through driving the rotation of the cradle head, this design can track and monitor moving objective in complicated conditions and keep objective in the controlling pictures until objective is lost.
The paper described this from two aspects:
1. Algorithm Analysis and Design:
This section mainly completes an efficient separation algorithm for MJPEG images and gives a new background extraction algorithm based on the frame difference algorithm. It adopts separation corner positioned on moving objective to describe moving objective through detailed analysis and study of minimum values of the corner detection algorithm provided by the OpenCV library. Finally the article analyses and studies the LK's pyramid algorithm in detail, adopts the LK pyramid sparse optical flow algorithm provided by OpenCV library to find the current frame moving target, calculates the location in the current screen of the moving target'center. Then drives the cradle head movement, and makes the target object gradually approximate to the central screen, until the goal of moving objects is lost, and finally stops the movement of the cradle head. The above process of background extraction and objectives calculation will be cycled.
2. Porting and Optimizing Existing Code
As the final execution platform is the PXA270 board developed by 51 Board company and this design takes use of a large number of complex algorithms, which entails the optimization of the code by Wireless MMX technology in order to maximize the performance of the processor. The cross compiler this company provided does not support Wireless MMX instruction set and the operating system is kernel Linux2.4, neither are able to play the best performance.
The main part of the completion of two elements
1)、Establishing PXA270 platform
First I adopt the cross-tool provided by GUN and generate a compiler which is specified for the PXA270 and support Wireless MMX and EABI (The Embedded Application Binary Interface), then I port Linux2.6.28 kernel to the PXA270 board, enable the support of operating system's floating-point simulator, WMMX technology, FrameBuffer and network. At last, I use the BusyBox to complete my file system, achieving the maximum optimization for the running system on the PXA270.
2)、Optimizing OpenCV libraries and code
Wireless WMMX technology proposed by Intel uses SIMD instruction set, it can computer at the same time 64-bit, up to 8 units of bytes data. Without WMMX instruction in code, the compiler is not smart to generate automatically assembly instructions supported by WMMX technology. In the code, background extraction, feature point calculation, LK's optical flow algorithm are consuming a large number of CPU time by matrix, floating-point calculations. I completed the optimization of the OpenCV library and my code after the detailed study of the WMMX Intel instruction set.
As the PXA270 processor does not support floating-point instructions, I uses floating-point emulator provided by the Linux kernel. The efficient of process executed in PXA270 can not compare with in x86 processor with support for floating-point instruction. With collecting 640*480 images in the PXA270 platform I complete the design at about 5 FPS in the processing performance after decrease of the image size in the internal processing procedures.
引文
[1]Intel(?) Wireless MMXTM Technology Developer Guide 2002
[2]Intel XScale(?) Core Developer's Manual 2004:94-95
[3]Intel(?) PXA27x Processor Family Developer's Manual 2006:1-5-1-6
[4]Linux内核官方网站http://www.kernel.org
[5]章毓晋.图像工程.第2版.清华大学出版社,2007年：358
[6]Intel(?) C++ Intrinsic Reference
[7]Intel(?) C++ Compiler for Linux Intrinsic Reference
[8]Rafael C. Gonzalez, Richard E. Woods数字图像处理.阮秋琦,阮宇智.第2版.电子工业出版社,2008
[9]Gary Bradski,Adrian Kaebler学习OpenCV中文版.于仕琪,刘瑞祯.清华大学出版社,2009
[10]OpenCV Documentation.2009
[11]孙潜.基于PXA270开发板的Linux的移植和研究.青岛大学.硕士.2008
[12]Aaron Wong. Building arm-linux toolchain for ARM/XScale-iWMMXt(new ABI).2009
[13]Intel(?) XScaleTM Microarchitecture Programmers Reference Manual
[14]C. Harris,M.J. Stephens.A Combined Corner and Edge Detector.Fourth Alvey Vision Conf., pp.147-151.1988
[15]Karim Yaghmour.Building Embedded Linux Systems.O'Reilly,2003
[16]W.Richard Stevens, Stephen A. Rago. Advanced Programming in the UNIX(?) Environment. Second Edition. Addison Wesley Professional,2005
[17]Andrew N.Sloss,Dominic Symes,Chris Wright.ARM嵌入式系统开发-软件设计与优化.沈建华.北京航空航天大学出版社,2005
[18]Karim Yagbmour构建嵌入式LINUX系统.韩存兵,龚波.中国电力出版社,2004
[19]Darwin yuan.Developing Your Own OS On IBM PC.2008
[20]Stanley B Lippman, Josee Lajoie.C++ Primer中文版.李师贤,蒋爱军,梅晓勇,林瑛.第四版.人民邮电出版社,2009
[21]李晓潮,焦李成,洪新华,卢潇.基于嵌入式SIMD处理器的音频解码优化.计算机工程.2009,第35卷(1)
[22]高卓.基于机器人视觉的运动目标检测及跟踪算法研究.哈尔滨工程大学.硕士.2007
[23]查成东,王长松,巩宪锋,周家新.基于改进K-均值聚类算法的背景提取方法.计算机工程与设计.2007,第28卷(21)
[24]袁基炜,史忠科.一种快速运动目标的背景提取算法.计算机应用研究.2004,第8期
[25]罗万福,艾斯卡尔.视频监控系统中的背景提取算法.视频技术与应用.2006,第12期
[26]丁小飞.复杂场景下背景提取与自适应更新算法在DSP上的实现.南京邮电大学.硕士.2007
[27]赖太国.基于PCI视频监控系统的背景提取及更新算法研究及其实现.电子科技大学.硕士.2009：29
[28]吴众山,雷蕴奇,吴绿芳.一种实用的背景提取与更新算法.厦门大学学报.2008,第47卷(3)
[29]林卉,赵长胜,舒宁.基于Canny算子的边缘检测及评价.黑龙江工程学院学报.2003,第17卷(2)
[30]Jianbo Shi,Carlo Tomasi.Good Features to Track.IEEE.1994
[31]F.施依德.数值分析.罗亮生,包雪松,王国英.第二版.科学出版社,2002年
[32]杜春雷.ARM体系结构与编程.清华大学出版社.2003
[33]孟浩.基于机器人视觉的运动目标检测及跟踪算法研究.哈尔滨工程大学.硕士.2007
[34]J. Y. Bouguet. Pyramidal implementation of the Lucas Kanade Feature Tracker Description of the algorithm. Intel Corporation Microprocessor Research Labs.2003
[35]韩斌,周增雨,王士同.改进的亚像素级快速角点检测算法.江苏科技大学学报.2009,第23卷(2)
[36]ARM Architecture Reference Manual
[37]Robert Mecklenburg.GNU Make项目管理O'Reilly Taiwan公司.第三版.2006
[38]郑南宁.计算机视觉与模式识别.国防工业出版社.1998
[39]王晓东.算法设计与分析.清华大学出版社.2003
[40]西奥多里蒂斯.模式识别.李晶皎.第四版.电子工业出版社,2010
[41]Nelson H.F. Beebe, Arnold Robbins. Classic Shell Scripting. O'Reilly.2005
[42]同济大学数学系.高等数学.第六版.高等教育出版社,2007
[43]盛骤.概率论与数理统计.第三版.高等教育出版社,2001
[44]李俊.嵌入式Linux设备驱动开发详解.人民邮电出版社,2008：155～166
[45]ARM官方站点http://www.arm.com/
[46]OpenCV官方站点http://opencv.willowgarage.com/wiki/
[47]OpenCV中文站点http://www.opencv.org.cn/index.php/首页
[48]S.M. Smith,J.M. Brady.SUSAN-A NEW Approach to Low Level Image Processing. Int'l J. Computer Vision,1995. vol.23, no.1
[49]GUN官方站点http://www.gnu.org/
[50]BusyBox官方站点http://www.busvbox.net/
[51]赵万金,龚声蓉,刘纯平,沈项军.一种自适应的Harris角点检测算法.计算机工程.2008.第34卷(10)
[2]Intel XScale(?) Core Developer's Manual 2004:94-95
[3]Intel(?) PXA27x Processor Family Developer's Manual 2006:1-5-1-6
[4]Linux内核官方网站http://www.kernel.org
[5]章毓晋.图像工程.第2版.清华大学出版社,2007年：358
[6]Intel(?) C++ Intrinsic Reference
[7]Intel(?) C++ Compiler for Linux Intrinsic Reference
[8]Rafael C. Gonzalez, Richard E. Woods数字图像处理.阮秋琦,阮宇智.第2版.电子工业出版社,2008
[9]Gary Bradski,Adrian Kaebler学习OpenCV中文版.于仕琪,刘瑞祯.清华大学出版社,2009
[10]OpenCV Documentation.2009
[11]孙潜.基于PXA270开发板的Linux的移植和研究.青岛大学.硕士.2008
[12]Aaron Wong. Building arm-linux toolchain for ARM/XScale-iWMMXt(new ABI).2009
[13]Intel(?) XScaleTM Microarchitecture Programmers Reference Manual
[14]C. Harris,M.J. Stephens.A Combined Corner and Edge Detector.Fourth Alvey Vision Conf., pp.147-151.1988
[15]Karim Yaghmour.Building Embedded Linux Systems.O'Reilly,2003
[16]W.Richard Stevens, Stephen A. Rago. Advanced Programming in the UNIX(?) Environment. Second Edition. Addison Wesley Professional,2005
[17]Andrew N.Sloss,Dominic Symes,Chris Wright.ARM嵌入式系统开发-软件设计与优化.沈建华.北京航空航天大学出版社,2005
[18]Karim Yagbmour构建嵌入式LINUX系统.韩存兵,龚波.中国电力出版社,2004
[19]Darwin yuan.Developing Your Own OS On IBM PC.2008
[20]Stanley B Lippman, Josee Lajoie.C++ Primer中文版.李师贤,蒋爱军,梅晓勇,林瑛.第四版.人民邮电出版社,2009
[21]李晓潮,焦李成,洪新华,卢潇.基于嵌入式SIMD处理器的音频解码优化.计算机工程.2009,第35卷(1)
[22]高卓.基于机器人视觉的运动目标检测及跟踪算法研究.哈尔滨工程大学.硕士.2007
[23]查成东,王长松,巩宪锋,周家新.基于改进K-均值聚类算法的背景提取方法.计算机工程与设计.2007,第28卷(21)
[24]袁基炜,史忠科.一种快速运动目标的背景提取算法.计算机应用研究.2004,第8期
[25]罗万福,艾斯卡尔.视频监控系统中的背景提取算法.视频技术与应用.2006,第12期
[26]丁小飞.复杂场景下背景提取与自适应更新算法在DSP上的实现.南京邮电大学.硕士.2007
[27]赖太国.基于PCI视频监控系统的背景提取及更新算法研究及其实现.电子科技大学.硕士.2009：29
[28]吴众山,雷蕴奇,吴绿芳.一种实用的背景提取与更新算法.厦门大学学报.2008,第47卷(3)
[29]林卉,赵长胜,舒宁.基于Canny算子的边缘检测及评价.黑龙江工程学院学报.2003,第17卷(2)
[30]Jianbo Shi,Carlo Tomasi.Good Features to Track.IEEE.1994
[31]F.施依德.数值分析.罗亮生,包雪松,王国英.第二版.科学出版社,2002年
[32]杜春雷.ARM体系结构与编程.清华大学出版社.2003
[33]孟浩.基于机器人视觉的运动目标检测及跟踪算法研究.哈尔滨工程大学.硕士.2007
[34]J. Y. Bouguet. Pyramidal implementation of the Lucas Kanade Feature Tracker Description of the algorithm. Intel Corporation Microprocessor Research Labs.2003
[35]韩斌,周增雨,王士同.改进的亚像素级快速角点检测算法.江苏科技大学学报.2009,第23卷(2)
[36]ARM Architecture Reference Manual
[37]Robert Mecklenburg.GNU Make项目管理O'Reilly Taiwan公司.第三版.2006
[38]郑南宁.计算机视觉与模式识别.国防工业出版社.1998
[39]王晓东.算法设计与分析.清华大学出版社.2003
[40]西奥多里蒂斯.模式识别.李晶皎.第四版.电子工业出版社,2010
[41]Nelson H.F. Beebe, Arnold Robbins. Classic Shell Scripting. O'Reilly.2005
[42]同济大学数学系.高等数学.第六版.高等教育出版社,2007
[43]盛骤.概率论与数理统计.第三版.高等教育出版社,2001
[44]李俊.嵌入式Linux设备驱动开发详解.人民邮电出版社,2008：155～166
[45]ARM官方站点http://www.arm.com/
[46]OpenCV官方站点http://opencv.willowgarage.com/wiki/
[47]OpenCV中文站点http://www.opencv.org.cn/index.php/首页
[48]S.M. Smith,J.M. Brady.SUSAN-A NEW Approach to Low Level Image Processing. Int'l J. Computer Vision,1995. vol.23, no.1
[49]GUN官方站点http://www.gnu.org/
[50]BusyBox官方站点http://www.busvbox.net/
[51]赵万金,龚声蓉,刘纯平,沈项军.一种自适应的Harris角点检测算法.计算机工程.2008.第34卷(10)