基于FPGA的实时视频信号处理系统的设计
|
摘要
实时视频信号处理的实时性与自动聚焦技术休戚相关。电子技术和数字图像处理技术的发展,使自动聚焦技术向基于数字图像处理的方向发展,基于图像高频分量值的自动聚焦技术应用而生。视频信号的格式转换和实现问题也是当前研究的重点。本文在着重介绍一种一体化摄像机完成自动聚焦功能的同时,完成了实时视频信号处理系统的设计,其内容包括以下几点：
(1)自动聚焦分为聚焦调节和聚焦监测两部分。聚焦调节又包括变倍跟踪和聚焦搜索。变倍跟踪即对图像进行变倍时,聚焦电机要根据变焦跟踪曲线跟随变倍电机转动相应的步数；聚焦搜索即根据搜索算法在变倍跟踪结束后再进行一次自动聚焦。聚焦监测的主要作用是判断是否需要重启自动聚焦以及重启自动聚焦的条件。
(2)对灰度差分法进行了改进,增加了相邻像素的比较,这样就增大了聚焦评价函数的绝对值,提高了自动聚焦的精准度。
(3)提出了一种新的聚焦搜索算法,其主要特点是：步长根据聚焦评价函数的斜率的倒数来确定,且在两次比较聚焦评价函数值后再确定最佳聚焦点。这样不仅提高了聚焦速度,避免落入局部最大值圈套,而且对视频信号变化速度快引起的聚焦失败有一定的抵制作用。
(4)对805lIP核进行了详细分析论证,设计了基于FPGA的8051IP核,各个模块用VHDL语言实现,并用Modelsim进行了仿真验证。
(5)电机驱动和聚焦搜索程序由C语言完成,在KeilC环境下编译,并最终生成.coe文件下载到805lIP核中驱动电机完成变倍跟踪和聚焦搜索。
(6)对视频信号进行了后端处理,包括图像放大、帧率提升和颜色空间转换,并用VHDL语言实现。
(7)在完成了系统软件的描述后,进行了系统硬件的搭建,包括FPGA板、视频信号采集板和镜头之间的连接。
整个视频信号处理系统在XUPV5-LX110T FPGA上验证,变倍放大后具有自动聚焦功能,自动聚焦执行时间在2秒以内,视频信号的分辨率由736*576提升到1024*768,格式由8位YCbCr信号变为24位的RGB信号。该系统还可以进行扩展,例如对图像进行压缩编码、解码、本地存储及以太网传输等。
It has close relationship between the real-time of real-time video signal processing system and auto-focus technology. As the development of electronic technology and digital image processing technology, the direction of auto-focus is based on digital image processing technology. Auto-focus based on high frequency component was born. Video signal format conversion and implementation are also focused in present. This paper descript an integrated camera that completes the auto-focus function, and devise a system of real-time video signal processing, its content can be summarized as the follows:
(1)Auto-focus is divided into two parts:focus adjustment and focus detection. Focus adjustment contains zoom tracking and focus searching. Zoom tracking:The focus motor rotates corresponding number of steps according to the zoom tracking curve when the zoom motor changes the zoom of the images. Focus searching:Focus automatically again according to the search algorithm after zoom tracking. The main work of focus detection is determining whether restart auto-focus and the condition of auto-focus.
(2)Improves the gray scale differential algorithm and Increase the number of adjacent pixels comparison to increase the absolute value of focus evaluation function and improve the accuracy of the auto-focus.
(3)Describe an improved auto-focus searching algorithm. Its main features can be described as follow:step is dominated by the reciprocal of the slope of the focusing evaluation function curve, and the best focus point is fixed when compare the focusing evaluation function value two times. This will not only raise the speed of auto-focus, avoid falling into local maximum, but also resist the failure of focus because of the quick change of the vidio signal.
(4)Analysis and demonstrate a8051IPcore and design it based on FPGA. Its every module is realized by VHDL language and simulated by Modelsim。
(5)Motor-driven and focus searching code are write by C language and compiled in Keil C environment. At last, we Generate.coe file and download it into805IIP core to complete the zoom tracking and focus searching.
(6)Process video signals at the end of the system which includes color-space conversion, frame rate ascension and image magnification and realize them by VHDL language。
(7)We get the hardware structure of the system after software description. Hardware structure contains the connection among the FPGA board, video signal acquisition board and lens.
The whole video signal processing system is tested and verified on XUPV5-LX110T FPGA board. It has the function of focus when zoom finish and its execution time of auto-focus is less than1second. The resolution of the video signal is elevated from736*576up to1024*768.The frame of the video signal is elevated from12Hz up to60Hz.The format of the video signal is changed from8-bit YCbCr into24-bit RGB. This system can be expanded, such as the image compression encoding/decoding, local storage and Ethernet transport etc.
(1)自动聚焦分为聚焦调节和聚焦监测两部分。聚焦调节又包括变倍跟踪和聚焦搜索。变倍跟踪即对图像进行变倍时,聚焦电机要根据变焦跟踪曲线跟随变倍电机转动相应的步数；聚焦搜索即根据搜索算法在变倍跟踪结束后再进行一次自动聚焦。聚焦监测的主要作用是判断是否需要重启自动聚焦以及重启自动聚焦的条件。
(2)对灰度差分法进行了改进,增加了相邻像素的比较,这样就增大了聚焦评价函数的绝对值,提高了自动聚焦的精准度。
(3)提出了一种新的聚焦搜索算法,其主要特点是：步长根据聚焦评价函数的斜率的倒数来确定,且在两次比较聚焦评价函数值后再确定最佳聚焦点。这样不仅提高了聚焦速度,避免落入局部最大值圈套,而且对视频信号变化速度快引起的聚焦失败有一定的抵制作用。
(4)对805lIP核进行了详细分析论证,设计了基于FPGA的8051IP核,各个模块用VHDL语言实现,并用Modelsim进行了仿真验证。
(5)电机驱动和聚焦搜索程序由C语言完成,在KeilC环境下编译,并最终生成.coe文件下载到805lIP核中驱动电机完成变倍跟踪和聚焦搜索。
(6)对视频信号进行了后端处理,包括图像放大、帧率提升和颜色空间转换,并用VHDL语言实现。
(7)在完成了系统软件的描述后,进行了系统硬件的搭建,包括FPGA板、视频信号采集板和镜头之间的连接。
整个视频信号处理系统在XUPV5-LX110T FPGA上验证,变倍放大后具有自动聚焦功能,自动聚焦执行时间在2秒以内,视频信号的分辨率由736*576提升到1024*768,格式由8位YCbCr信号变为24位的RGB信号。该系统还可以进行扩展,例如对图像进行压缩编码、解码、本地存储及以太网传输等。
It has close relationship between the real-time of real-time video signal processing system and auto-focus technology. As the development of electronic technology and digital image processing technology, the direction of auto-focus is based on digital image processing technology. Auto-focus based on high frequency component was born. Video signal format conversion and implementation are also focused in present. This paper descript an integrated camera that completes the auto-focus function, and devise a system of real-time video signal processing, its content can be summarized as the follows:
(1)Auto-focus is divided into two parts:focus adjustment and focus detection. Focus adjustment contains zoom tracking and focus searching. Zoom tracking:The focus motor rotates corresponding number of steps according to the zoom tracking curve when the zoom motor changes the zoom of the images. Focus searching:Focus automatically again according to the search algorithm after zoom tracking. The main work of focus detection is determining whether restart auto-focus and the condition of auto-focus.
(2)Improves the gray scale differential algorithm and Increase the number of adjacent pixels comparison to increase the absolute value of focus evaluation function and improve the accuracy of the auto-focus.
(3)Describe an improved auto-focus searching algorithm. Its main features can be described as follow:step is dominated by the reciprocal of the slope of the focusing evaluation function curve, and the best focus point is fixed when compare the focusing evaluation function value two times. This will not only raise the speed of auto-focus, avoid falling into local maximum, but also resist the failure of focus because of the quick change of the vidio signal.
(4)Analysis and demonstrate a8051IPcore and design it based on FPGA. Its every module is realized by VHDL language and simulated by Modelsim。
(5)Motor-driven and focus searching code are write by C language and compiled in Keil C environment. At last, we Generate.coe file and download it into805IIP core to complete the zoom tracking and focus searching.
(6)Process video signals at the end of the system which includes color-space conversion, frame rate ascension and image magnification and realize them by VHDL language。
(7)We get the hardware structure of the system after software description. Hardware structure contains the connection among the FPGA board, video signal acquisition board and lens.
The whole video signal processing system is tested and verified on XUPV5-LX110T FPGA board. It has the function of focus when zoom finish and its execution time of auto-focus is less than1second. The resolution of the video signal is elevated from736*576up to1024*768.The frame of the video signal is elevated from12Hz up to60Hz.The format of the video signal is changed from8-bit YCbCr into24-bit RGB. This system can be expanded, such as the image compression encoding/decoding, local storage and Ethernet transport etc.
引文
[1]李逢雷.基于FPGA的自动聚焦系统设计[D].硕士学位论文,太原理工大学,2008.
[2]张博,李诚,刘剑,刘鑫.实时视频信号处理平台的设计[J].OpenHW09太原理工大学参赛项目,太原理工大学,2010.
[3]张荐.自动聚焦控制集成电路的关键技术研究[D],硕士学位论文,太原理工大学,2010.
[4]Yao Wang, Jorn Ostermann, Ya-Qin Zhang, Video Processing and Communications[M], New Jersey:Pearson Education,2003
[5]胡凤萍.视频自动聚焦方法研究与实现[D].硕士学位论文,西安电子科技大学,2008.
[6]王瑞,游志宇,杜杨,王军.MC8051单片机IP核的FOGA实现与应用[J].电子设计工程,2009,17(1)：57-63.
[7]翟呈祥.基于FPGA的8051单片机IP核设计及应用[D].硕士学位论文,太原理工大学,2007.
[8]邢强.IP软核复用技术研究[D].硕士学位论文,北京交通大学,2006.
[9]侯伯亨,刘凯,顾新等VHDL硬件描述语言与数字逻辑电路设计[M],西安,西安电子科技大学出版社.
[10]汪德彪.MCS-51单片机原理及接口技术[M].北京：电子工业出版社,2009.
[11]邓东.理解摄像机[A].小镜子互动传媒出版发行,2008.
[12]杨涛,左勇,程晓梅.基于图像处理的自动聚焦技术及应用[J],计算机仿真,2009,7(26),256-259.
[13]张博,张刚,程永强等.一体化摄像机中变焦控制系统的设计[J].电视技术,2010,(4)：18-20
[14]胡凤萍,常义林,马彦卓,赵光耀.视频自动聚焦的实现研究[J].光子学报,2010,39(10)：1901—1906.
[15]周贤,姜威.基于边缘能量的自动聚焦算法[J].光学技术,2006,32(2)：213-218.
[16]Kang-Sun Choi, Sung-Jae eKo. A New Auto-focusing Technique Using The Frequency Selective Weighted Median Filter For Video Cameras.
[17]E-Ho Lee, Kum, Byung-Deok Nam et al. Implementation of Passive Automatic Focusing Algorithm For Digital Still Camera[J]. IEEE Transactions On Consumer Electronics,2004,13(6):256-259.
[18]郭军,曾文函,谢铁邦.基于熵函数的快速自动聚焦方法[J].计量技术,2003,11：204-207.
[19]任四刚,李见为,谢利利.基于灰度差分法的自动调焦技术[J].光电工程,2003,30(2)：53—55.
[20]姜威,高赞,朱孔凤.一种改进的自动聚焦算法[J].山东大学学报(理学版)2006,4l(5)：120-123.
[21]黄剑琪,冯华君,徐之海等.边缘特征增强算法和小波分析在精确调焦中的应用[J].光子学报,2000,29(10)：932-934.
[22]营维乐,姜威,周贤.一种基于小波变换的数字图像自动聚焦算法[J].山东大学学报,2004,34(6)：38-40.
[23]郑玉珍.自动对焦中的优化爬山搜索算法[J].浙江科技学院学报,2005,17(3)：17l-174.
[24]官光勇,何文忠,高旭辉.红外系统中自动调焦爬山搜索算法的优化设计[J].激光与红外,2007,37(11)：1213—1215.
[25]郑静,张起贵,王朋.一种改进的自动聚焦搜索算法及其实现[J].电脑开发与应用,2010,23(1)：26-27.
[26]潘涛,程耕国.步进电机控制器的FPGA实现[J].现代电子术,2009,(1)：148—150.
[27]赵轶彦.步进电机的智能控制在自动聚焦中的应用研究[D].硕士学位论文,太原理工大学,2008.
[28]罗钧,孙立,闵志盛.变倍跟踪曲线在对焦中的应用[J].光学精密工程,2011,19(10)：2326-2332.
[29]江凤莲,曾志宏.图像插值放大方法的研究与应用[J].制造业自动化,2011,33(1)：185—186.
[30]赵海峰,周永飞,黄子强.图像放大算法比较研究[J].现代电子技术,2010,(24)：33—39.
[31]苗莎,郑啸薇.双线性插值图像放大并行算法的设计与实现[J].微电子学与计算机,2011,28(11)：182—184.
[32]Blu T,Thevenaz P,Uneser M.Linear Interpolation Revitalized[J].IEEE Transactions on Image Processing,2004,13(5):710-719.
[33]马学超.基于小波变换的图像放大法研究[J].产业与科技论坛,2011,10(16)：88-89.
[34]江巨浪.基于改进Catmull-Rom样条的图像缩放算法[J].计算机技术与发展,2007,17(4)：211—217.
[35]林福宗.多媒体技术教程[M].北京：清华大学出版社,2009
[36]张陌.基于IZAS图像放大算法的SONY47万像素数字视频展台系统[D].硕士学位论文,太原理工大学,2008.
[37]冷星星,滕奇志,卿粼波等.基于FPGA视频处理系统的颜色空间转换[J].PLD CPLD FPGA应用,2008,24(8—2)：219-225.
[38]谢斌,张小红,肖玲玲.基于颜色空间转换的自适应视频多重水印算法[J].数字视频,2011,35(17)：15—18.
[39]Kenneth R.Castleman.Digital Image Processing[M].北京：电子工业出版社,2008.
[40]曹永健.基于VHDL语言的8051IP核的设计与验证研究[D].硕士学位论文,山东师范大学,2007.
[41]苏学颖.基于8051单片机IP软核的优化设计及应用研究[D].硕士学位论文,西华大学,2006.
[42]张林生.TSK微处理器IP核在FPGA中的实现与应用[J].工程技术,2011,(28)：49.
[43]薛迎春.基于FPGA的8051IP核的设计与验证研究[D].硕士学位论文,苏州大学,2010.
[44]黄建军,王金明,于坚.基于FPGA的8051IP核的设计与应用[J].军事通信技术,2008,29(2)：86-89.
[45]杨先文,李峥.在FPGA上对OC8051IP核的修改与测试[J].新器件新技术,2009,(11)：44-46.
[46]http://www.xilinx.com/products/silicon_solutions/fpgas/virtex/virtex5
[47]www.docin.com>通信/电子>电子设计>TrueView5725 datasheet.
[2]张博,李诚,刘剑,刘鑫.实时视频信号处理平台的设计[J].OpenHW09太原理工大学参赛项目,太原理工大学,2010.
[3]张荐.自动聚焦控制集成电路的关键技术研究[D],硕士学位论文,太原理工大学,2010.
[4]Yao Wang, Jorn Ostermann, Ya-Qin Zhang, Video Processing and Communications[M], New Jersey:Pearson Education,2003
[5]胡凤萍.视频自动聚焦方法研究与实现[D].硕士学位论文,西安电子科技大学,2008.
[6]王瑞,游志宇,杜杨,王军.MC8051单片机IP核的FOGA实现与应用[J].电子设计工程,2009,17(1)：57-63.
[7]翟呈祥.基于FPGA的8051单片机IP核设计及应用[D].硕士学位论文,太原理工大学,2007.
[8]邢强.IP软核复用技术研究[D].硕士学位论文,北京交通大学,2006.
[9]侯伯亨,刘凯,顾新等VHDL硬件描述语言与数字逻辑电路设计[M],西安,西安电子科技大学出版社.
[10]汪德彪.MCS-51单片机原理及接口技术[M].北京：电子工业出版社,2009.
[11]邓东.理解摄像机[A].小镜子互动传媒出版发行,2008.
[12]杨涛,左勇,程晓梅.基于图像处理的自动聚焦技术及应用[J],计算机仿真,2009,7(26),256-259.
[13]张博,张刚,程永强等.一体化摄像机中变焦控制系统的设计[J].电视技术,2010,(4)：18-20
[14]胡凤萍,常义林,马彦卓,赵光耀.视频自动聚焦的实现研究[J].光子学报,2010,39(10)：1901—1906.
[15]周贤,姜威.基于边缘能量的自动聚焦算法[J].光学技术,2006,32(2)：213-218.
[16]Kang-Sun Choi, Sung-Jae eKo. A New Auto-focusing Technique Using The Frequency Selective Weighted Median Filter For Video Cameras.
[17]E-Ho Lee, Kum, Byung-Deok Nam et al. Implementation of Passive Automatic Focusing Algorithm For Digital Still Camera[J]. IEEE Transactions On Consumer Electronics,2004,13(6):256-259.
[18]郭军,曾文函,谢铁邦.基于熵函数的快速自动聚焦方法[J].计量技术,2003,11：204-207.
[19]任四刚,李见为,谢利利.基于灰度差分法的自动调焦技术[J].光电工程,2003,30(2)：53—55.
[20]姜威,高赞,朱孔凤.一种改进的自动聚焦算法[J].山东大学学报(理学版)2006,4l(5)：120-123.
[21]黄剑琪,冯华君,徐之海等.边缘特征增强算法和小波分析在精确调焦中的应用[J].光子学报,2000,29(10)：932-934.
[22]营维乐,姜威,周贤.一种基于小波变换的数字图像自动聚焦算法[J].山东大学学报,2004,34(6)：38-40.
[23]郑玉珍.自动对焦中的优化爬山搜索算法[J].浙江科技学院学报,2005,17(3)：17l-174.
[24]官光勇,何文忠,高旭辉.红外系统中自动调焦爬山搜索算法的优化设计[J].激光与红外,2007,37(11)：1213—1215.
[25]郑静,张起贵,王朋.一种改进的自动聚焦搜索算法及其实现[J].电脑开发与应用,2010,23(1)：26-27.
[26]潘涛,程耕国.步进电机控制器的FPGA实现[J].现代电子术,2009,(1)：148—150.
[27]赵轶彦.步进电机的智能控制在自动聚焦中的应用研究[D].硕士学位论文,太原理工大学,2008.
[28]罗钧,孙立,闵志盛.变倍跟踪曲线在对焦中的应用[J].光学精密工程,2011,19(10)：2326-2332.
[29]江凤莲,曾志宏.图像插值放大方法的研究与应用[J].制造业自动化,2011,33(1)：185—186.
[30]赵海峰,周永飞,黄子强.图像放大算法比较研究[J].现代电子技术,2010,(24)：33—39.
[31]苗莎,郑啸薇.双线性插值图像放大并行算法的设计与实现[J].微电子学与计算机,2011,28(11)：182—184.
[32]Blu T,Thevenaz P,Uneser M.Linear Interpolation Revitalized[J].IEEE Transactions on Image Processing,2004,13(5):710-719.
[33]马学超.基于小波变换的图像放大法研究[J].产业与科技论坛,2011,10(16)：88-89.
[34]江巨浪.基于改进Catmull-Rom样条的图像缩放算法[J].计算机技术与发展,2007,17(4)：211—217.
[35]林福宗.多媒体技术教程[M].北京：清华大学出版社,2009
[36]张陌.基于IZAS图像放大算法的SONY47万像素数字视频展台系统[D].硕士学位论文,太原理工大学,2008.
[37]冷星星,滕奇志,卿粼波等.基于FPGA视频处理系统的颜色空间转换[J].PLD CPLD FPGA应用,2008,24(8—2)：219-225.
[38]谢斌,张小红,肖玲玲.基于颜色空间转换的自适应视频多重水印算法[J].数字视频,2011,35(17)：15—18.
[39]Kenneth R.Castleman.Digital Image Processing[M].北京：电子工业出版社,2008.
[40]曹永健.基于VHDL语言的8051IP核的设计与验证研究[D].硕士学位论文,山东师范大学,2007.
[41]苏学颖.基于8051单片机IP软核的优化设计及应用研究[D].硕士学位论文,西华大学,2006.
[42]张林生.TSK微处理器IP核在FPGA中的实现与应用[J].工程技术,2011,(28)：49.
[43]薛迎春.基于FPGA的8051IP核的设计与验证研究[D].硕士学位论文,苏州大学,2010.
[44]黄建军,王金明,于坚.基于FPGA的8051IP核的设计与应用[J].军事通信技术,2008,29(2)：86-89.
[45]杨先文,李峥.在FPGA上对OC8051IP核的修改与测试[J].新器件新技术,2009,(11)：44-46.
[46]http://www.xilinx.com/products/silicon_solutions/fpgas/virtex/virtex5
[47]www.docin.com>通信/电子>电子设计>TrueView5725 datasheet.