基于DSP的视频自动聚焦系统研究
|
摘要
随着数字图像处理技术的日益发展,自动聚焦技术被广泛的应用于数码相机、数码摄像机以及显微镜成像等领域。获得清晰的视频图像是后续视频处理、视频分析的基础,因此实现视频图像快速、精确、智能化的聚焦显得十分重要。但现有的基于图像的自动聚焦算法都是针对于单一主体场景,并且都未考虑视频聚焦的平滑性问题。同时由于现有的聚焦算法都不能很好的分割视频场景中前景和背景部分,导致场景中聚焦主体的成像质量不高,尤其在多主体场景下将会导致误聚焦的出现。基于上述研究背景本文展开了基于DSP的自动聚焦系统的研究,主要工作和研究成果如下:
1.在单一主体场景聚焦中,为了满足视频聚焦过程平滑性的要求,提出了自适应平滑聚焦算法。首先建立了基于图像的自动聚焦系统模型,该模型主要包含清晰度计算和聚焦搜索算法两个模块。为了使聚焦主体包含图像主体,采用了基于边缘检测的聚焦区域选择方法,并结合梯度阈值的清晰度计算方法对图像主体的清晰度值进行正确评价。为了保证聚焦过程的平滑性,提出了自适应聚焦搜索算法,该算法通过清晰度的变化程度来自适应调整搜索步长,从而实现快速、精确、平滑的视频聚焦。
2.在多主体场景聚焦中,由于清晰度曲线呈现多峰性,使得现有的聚焦算法都会陷入局部极值导致聚焦结果的不确定性。本文通过视频之间的关系来动态的搜寻视频场景中的聚焦主体,提出了多主体聚焦算法。该算法包括图像快速清晰度计算和多主体聚焦搜索算法。为了满足嵌入式设备实时性的要求,采用了基于Sobel算子清晰度计算方法。同时在聚焦搜索中,一种基于近景优先的聚焦搜索策略能够保证镜头聚焦到最近主体位置。
3.在ADI Blanckfin DSP平台上,对自动聚焦系统进行了实现。首先是基于DSP的聚焦系统架构设计,并在此基础上对视频采集驱动和镜头控制驱动进行了实现。接着对上述两个的聚焦算法进行了移植和优化,包括算法整体架构的实现、算法移植优化和数据流优化。最后是对整个聚焦系统进行了实际平台的搭建并给出了具体硬件型号和实物图。
With the development of digital image processing technology, the auto-focusing (AF) techniques have been widely applied in many imaging instruments such as camera, video and microscope. The achievement of the distinct image is the fundament of the image processing and the vision analyzing. So the fast auto-focusing with high accuracy becomes important. How to achieve smooth auto-focus for video applications is a chief problem in the auto-focusing techniques. Moreover, it is difficult to obtain good performance when there are multiple objects in the scene. From this viewpoint, this paper presents an auto-focusing system based on DSP. The main contents and achievements of the whole research are introduced as follows.
To achieve the smoonth auto-focus for the video applications, a fast and high accuracy auto-focusing algorithm with self-adaptive steps is proposed. Firstly, the model of the auto-focus system based on images is introduced, which is composed of the image sharpness measure and the search strategy. The region selection method based on edge detection is used to improve the accuracy of the image sharpenss measure. And a weighted gradient histogram based threshold selection method is used to reduce the influence of background noises. Then the smooth auto-focus search strategy with self-adaptive steps is designed according to the change of the image sharpness, which has good performance on speed, sensitivity of interference and smoothness of auto-focus search.
In the multi-objective situation, the image sharpness curve has multiple peaks, and the traditional AF algorithms are not robust due to the local maximum trapping. To solve this problem, an image based passive auto-focusing algorithm for the multi-objective situation is proposed. The algorithm is composed of the image sharpness measure and the search strategy. To meet real time requirement, an effective and efficient image sharpness calculation based on Sobel operator is proposed. During the multi-objective search, a near-object priority search method is presented to search the real peak. The method will drive the lens focus on the nearest object automatically by detecting sharpness changes in the sub-windows.
At last, the auto-focusing system is implemented on the ADI Blackfin DSP. Firstly, the design principle of hardware and software is presented. Then the drives for video capture and lens control are implemented. Moreover, the auto-focusing algorithm is transported to the DSP, and the data flow optimization and the memory optimization are used to improve the performance. Finally, the physical parts of the auto-focus system are illustrated in detail.
1.在单一主体场景聚焦中,为了满足视频聚焦过程平滑性的要求,提出了自适应平滑聚焦算法。首先建立了基于图像的自动聚焦系统模型,该模型主要包含清晰度计算和聚焦搜索算法两个模块。为了使聚焦主体包含图像主体,采用了基于边缘检测的聚焦区域选择方法,并结合梯度阈值的清晰度计算方法对图像主体的清晰度值进行正确评价。为了保证聚焦过程的平滑性,提出了自适应聚焦搜索算法,该算法通过清晰度的变化程度来自适应调整搜索步长,从而实现快速、精确、平滑的视频聚焦。
2.在多主体场景聚焦中,由于清晰度曲线呈现多峰性,使得现有的聚焦算法都会陷入局部极值导致聚焦结果的不确定性。本文通过视频之间的关系来动态的搜寻视频场景中的聚焦主体,提出了多主体聚焦算法。该算法包括图像快速清晰度计算和多主体聚焦搜索算法。为了满足嵌入式设备实时性的要求,采用了基于Sobel算子清晰度计算方法。同时在聚焦搜索中,一种基于近景优先的聚焦搜索策略能够保证镜头聚焦到最近主体位置。
3.在ADI Blanckfin DSP平台上,对自动聚焦系统进行了实现。首先是基于DSP的聚焦系统架构设计,并在此基础上对视频采集驱动和镜头控制驱动进行了实现。接着对上述两个的聚焦算法进行了移植和优化,包括算法整体架构的实现、算法移植优化和数据流优化。最后是对整个聚焦系统进行了实际平台的搭建并给出了具体硬件型号和实物图。
With the development of digital image processing technology, the auto-focusing (AF) techniques have been widely applied in many imaging instruments such as camera, video and microscope. The achievement of the distinct image is the fundament of the image processing and the vision analyzing. So the fast auto-focusing with high accuracy becomes important. How to achieve smooth auto-focus for video applications is a chief problem in the auto-focusing techniques. Moreover, it is difficult to obtain good performance when there are multiple objects in the scene. From this viewpoint, this paper presents an auto-focusing system based on DSP. The main contents and achievements of the whole research are introduced as follows.
To achieve the smoonth auto-focus for the video applications, a fast and high accuracy auto-focusing algorithm with self-adaptive steps is proposed. Firstly, the model of the auto-focus system based on images is introduced, which is composed of the image sharpness measure and the search strategy. The region selection method based on edge detection is used to improve the accuracy of the image sharpenss measure. And a weighted gradient histogram based threshold selection method is used to reduce the influence of background noises. Then the smooth auto-focus search strategy with self-adaptive steps is designed according to the change of the image sharpness, which has good performance on speed, sensitivity of interference and smoothness of auto-focus search.
In the multi-objective situation, the image sharpness curve has multiple peaks, and the traditional AF algorithms are not robust due to the local maximum trapping. To solve this problem, an image based passive auto-focusing algorithm for the multi-objective situation is proposed. The algorithm is composed of the image sharpness measure and the search strategy. To meet real time requirement, an effective and efficient image sharpness calculation based on Sobel operator is proposed. During the multi-objective search, a near-object priority search method is presented to search the real peak. The method will drive the lens focus on the nearest object automatically by detecting sharpness changes in the sub-windows.
At last, the auto-focusing system is implemented on the ADI Blackfin DSP. Firstly, the design principle of hardware and software is presented. Then the drives for video capture and lens control are implemented. Moreover, the auto-focusing algorithm is transported to the DSP, and the data flow optimization and the memory optimization are used to improve the performance. Finally, the physical parts of the auto-focus system are illustrated in detail.
引文
[1] Chen C-Y, Hwang R-C, Chen Y-J. A passive auto-focus camera control system [J]. Applied Soft Computing, 2010, 10(1): 296-303.
[2]黄家荣,张莉.基于图像识别技术的摄像机自动聚焦系统设计[J].四川师范大学学报, 2010, 33(3): 414-418.
[3] Gao Z, Jiang W, Zhu K-F, et al. Auto-focusing alogrithm based on roberts gradient [J]. Infrared and Laser Enginnering, 2006, 35(1): 117-211.
[4]冯华军,徐之海.红外主动式PSD测距系统[J].光电工程, 1999, 26(3): 42-46.
[5]陈国金.数字图像自动聚焦技术研究及系统实现[D].西安电子科技大学博士论文, 2007年.
[6]冯华君.反射能量法测距聚焦系统及其重叠设计法[J].光电工程, 1998, 25(2): 48-53.
[7]刘斌.基于图像技术的自动调焦方法研究与实现[D].浙江大学硕士论文, 2004年.
[8] Lee S-Y, Kumar Y, Cho J-M, et al. Enhanced autofocus algorithm using robust focus measure and fuzzy reasoning [J]. IEEE Transactions on Circuits and Systems for Video Technology. 2008, 18(9): 1237-1246.
[9]胡凤萍,常义林,马彦卓.视频自动聚焦的实现研究[J].光子学报, 2010, 39(10): 1901-1906.
[10] Batten C F. Auto focusing and astigmatism correction in the scanning electron microscope [D], M.Phil. Thesis, University of Cambridge, 2000.
[11]刘力双,孙双花,吕乃光.视觉系统中调焦函数的选择方法研究[J].北京机械工业学院学报, 2008, 23(1): 1-4.
[12]王勇,谭毅华,田金文.一种新的图像清晰度评价函数[J].武汉理工大学学报, 2007, 29(3): 124-126.
[13]朱孔凤.自动聚焦区域选择算法[J].安徽大学学报, 2009, 33(2): 31-33.
[14]张乐,姜威,高赞.数字图像一阶矩的自动聚焦区域选择算法[J].光学技术, 2008, 34(2): 163-169.
[15]王凤宇,王睿,张广军.基于DSP的自动聚焦系统[J].光电工程, 2007, 34(8): 134-138.
[16]王康泰,戴文战.一种基于Sobel算子和灰色关联度的图像边缘检测方法[J].计算机应用, 2006, 26(5): 1035-1047.
[17] Subbarao M, Tyan J K. Selecting the optimal focus measure for auto focusing and depth-from-focus [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(8): 864-870.
[18]丘前恕.显微照像的快速自动对焦技术[D].国立中央大学硕士论文, 2008年.
[19] Kautsky J, Flusser J, Zitova B, et al. A new wavelet-based measure of image focuses [J]. Pattern Recognition Letters, 2002, 23(14): 1785-1794.
[20]王义文,刘献礼,谢晖.基于小波变换的显微图像清晰度评价函数及3-D自动聚焦技术[J].光学精密工程, 2006, 14(6): 1063-1069
[21]周贤.自动聚焦算法研究与验证平台设计[D].山东大学硕士学位论文, 2006年.
[22]任四刚.图像式自动聚焦系统及其测量技术的研究[D].重庆大学硕士论文, 2002年.
[23]鲍歌堂.视觉检测系统中快速自动调焦技术的研究[D].上海交通大学硕士论, 2005年.
[24] Yeo T T E, Ong S H, Jayasooriah. Autofocusing for tissue microscopy [J]. Image and Vision Computing, 1993, 11(10): 629-639.
[25]胡涛,陈世哲,刘国栋.大范围自动调焦快速搜索算法[J].光电子激光, 2006, 17(4): 464-467.
[26] He J, Zhou R-Z, Hong Z-L. Modified fast climbing search auto-focus algorithm with adaptive step size searching technique for digital camera [J]. IEEE Transactions on Consumer Electronics, 2003, 49(2): 257-262.
[27]杨涛,左勇,陈晓梅.基于图像处理的自动聚焦技术及应用[J].计算机仿真, 2009, 26(7): 256-259.
[28] Chang H-C, Shih T-M, Chen N-Z, et al. A microscope system based on bevel-axial method auto-focus [J]. Optics and Lasers in Engineering, 2009, 47(5): 547-551.
[29]王勇,王典洪.基于图像清晰度的快速自动聚焦算法[J].计算机测量与控制, 2008, 16(3): 370-372.
[30] Lee J-S, Jung Y-Y, Kim B-S. An advanced video camera system with robust AF, AE, and AWB control [J]. IEEE Transactions on Consumer Electronics, 2001, 47(33): 694-699.
[31] Feng Q, Han K, Zhu X-C. An auto-focusing method for different object distance situation [J]. International Journal of Computer Science and Network Security, 2007, 7(6): 31-35.
[32]三星电机株式会社.自动聚焦方法及使用其的自动聚焦装置[P].中国专利: 200710079748.5, 2007-09-12.
[33]三洋电机株式会社.自动聚焦摄像机[P].中国专利: 98107054.X, 1998-11-25.
[34]盛云,秦亚杰,洪志良.一种用于数码视频相机的实时自动聚焦算法[J].光电工程, 2008, 35(8): 139-144.
[35]刘焕雨,熊文卓,万秋华.基于图像处理方法的自动调焦系统的研制[J].测试技术学报, 2007, 21(1): 13-16.
[36]李逢磊.基于FPGA的自动聚焦系统设计[D].太原理工大学, 2008年.
[37]潘广贞,陈华兵.工业摄像机自动聚焦系统设计[J].科技情报开发与经济, 2005, 15(9): 168-170.
[38]王超素,杨鸣,丁海波.一种基于视频信号处理的显微镜自动聚焦方法[J].宁波大学学报, 2008, 21(1): 11-14.
[39] Kehtarnavaz N, Oh H-J. Development and real-time implementation of a rule-based auto-focus algorithm [J]. Real-Time Imaging, 2003, 9(3): 197-203.
[40]王康泰,戴文战.一种基于Sobel算子和灰色关联度的图像边缘检测方法[J].计算机应用, 2006, 26(5): 1035-1047.
[41]张乐.自动聚焦系统精确性和实时性的研究[D].山东大学硕士论文, 2008年.
[42]高赞.自动聚焦评价函数的精确度和稳定性研究[D].山东大学硕士论文, 2007年.
[43]王欣,安志勇,杨瑞宁.基于图像清晰度评价函数的CCD摄像机自动调焦技术研究[J].长春理工大学学报, 2008, 31(1): 11-14.
[44] Chang H-C, Shih T-M, Chen N-Z. A microscope system based on bevel-axial method auto-focus [J]. Optics and Lasers in Engineering, 2009, 47(5): 547-551.
[45]袁珂,徐蔚鸿.基于图像清晰度评价的摄像头辅助调焦系统[J].光电工程, 2006, 33(1):141-144.
[46] ADI. ADSP-BF533 Blackfin Processor Hardware Reference [EB/OL]. http://www.analog.com/static/imported-files/processor_manuals/bf533_hwr_Rev3.4.pdf, 2009.
[47] ADI. Blackfin Processor Programming Reference [EB/OL]. http://www.analog.com/static/imported-files/processor_manuals/blackfin_pgr.ref.man.rev1.3.pdf, 2008.
[48] ADI. VisualDSP++ 4.5 User's Guide [EB/OL]. http://www.analog.com/static/imported-files/software_manuals_legacy/63652593555244616414545245_getstarted_man.pdf, 2006.
[49] ADI. ADSP-BF533 EZ-KIT Lite Evaluation System Manual [EB/OL]. http://www.analog.com/static/imported-files/eval_kit_manuals/288418312ADSP_BF533_EZ_KIT_Lite_Manual_Rev._3.1.pdf, 2007.
[50] ADI. VisualDSP++ 4.5 C/C++ Compiler and Library Manual for Blackfin Processors [EB/OL]. http://www.analog.com/static/imported-files/software_manuals_legacy/62641591695266839372613293690438645_Blackfin_comp_man.pdf, 2006.
[51] ADI. VisualDSP++ 4.5 Device Drivers and System Services Manual for Blackfin Processors [EB/OL]. http://www.analog.com/static/imported-files/software_manuals_legacy/6319559985497894170437445_systemservices_man_Rev21.pdf, 2006.
[2]黄家荣,张莉.基于图像识别技术的摄像机自动聚焦系统设计[J].四川师范大学学报, 2010, 33(3): 414-418.
[3] Gao Z, Jiang W, Zhu K-F, et al. Auto-focusing alogrithm based on roberts gradient [J]. Infrared and Laser Enginnering, 2006, 35(1): 117-211.
[4]冯华军,徐之海.红外主动式PSD测距系统[J].光电工程, 1999, 26(3): 42-46.
[5]陈国金.数字图像自动聚焦技术研究及系统实现[D].西安电子科技大学博士论文, 2007年.
[6]冯华君.反射能量法测距聚焦系统及其重叠设计法[J].光电工程, 1998, 25(2): 48-53.
[7]刘斌.基于图像技术的自动调焦方法研究与实现[D].浙江大学硕士论文, 2004年.
[8] Lee S-Y, Kumar Y, Cho J-M, et al. Enhanced autofocus algorithm using robust focus measure and fuzzy reasoning [J]. IEEE Transactions on Circuits and Systems for Video Technology. 2008, 18(9): 1237-1246.
[9]胡凤萍,常义林,马彦卓.视频自动聚焦的实现研究[J].光子学报, 2010, 39(10): 1901-1906.
[10] Batten C F. Auto focusing and astigmatism correction in the scanning electron microscope [D], M.Phil. Thesis, University of Cambridge, 2000.
[11]刘力双,孙双花,吕乃光.视觉系统中调焦函数的选择方法研究[J].北京机械工业学院学报, 2008, 23(1): 1-4.
[12]王勇,谭毅华,田金文.一种新的图像清晰度评价函数[J].武汉理工大学学报, 2007, 29(3): 124-126.
[13]朱孔凤.自动聚焦区域选择算法[J].安徽大学学报, 2009, 33(2): 31-33.
[14]张乐,姜威,高赞.数字图像一阶矩的自动聚焦区域选择算法[J].光学技术, 2008, 34(2): 163-169.
[15]王凤宇,王睿,张广军.基于DSP的自动聚焦系统[J].光电工程, 2007, 34(8): 134-138.
[16]王康泰,戴文战.一种基于Sobel算子和灰色关联度的图像边缘检测方法[J].计算机应用, 2006, 26(5): 1035-1047.
[17] Subbarao M, Tyan J K. Selecting the optimal focus measure for auto focusing and depth-from-focus [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(8): 864-870.
[18]丘前恕.显微照像的快速自动对焦技术[D].国立中央大学硕士论文, 2008年.
[19] Kautsky J, Flusser J, Zitova B, et al. A new wavelet-based measure of image focuses [J]. Pattern Recognition Letters, 2002, 23(14): 1785-1794.
[20]王义文,刘献礼,谢晖.基于小波变换的显微图像清晰度评价函数及3-D自动聚焦技术[J].光学精密工程, 2006, 14(6): 1063-1069
[21]周贤.自动聚焦算法研究与验证平台设计[D].山东大学硕士学位论文, 2006年.
[22]任四刚.图像式自动聚焦系统及其测量技术的研究[D].重庆大学硕士论文, 2002年.
[23]鲍歌堂.视觉检测系统中快速自动调焦技术的研究[D].上海交通大学硕士论, 2005年.
[24] Yeo T T E, Ong S H, Jayasooriah. Autofocusing for tissue microscopy [J]. Image and Vision Computing, 1993, 11(10): 629-639.
[25]胡涛,陈世哲,刘国栋.大范围自动调焦快速搜索算法[J].光电子激光, 2006, 17(4): 464-467.
[26] He J, Zhou R-Z, Hong Z-L. Modified fast climbing search auto-focus algorithm with adaptive step size searching technique for digital camera [J]. IEEE Transactions on Consumer Electronics, 2003, 49(2): 257-262.
[27]杨涛,左勇,陈晓梅.基于图像处理的自动聚焦技术及应用[J].计算机仿真, 2009, 26(7): 256-259.
[28] Chang H-C, Shih T-M, Chen N-Z, et al. A microscope system based on bevel-axial method auto-focus [J]. Optics and Lasers in Engineering, 2009, 47(5): 547-551.
[29]王勇,王典洪.基于图像清晰度的快速自动聚焦算法[J].计算机测量与控制, 2008, 16(3): 370-372.
[30] Lee J-S, Jung Y-Y, Kim B-S. An advanced video camera system with robust AF, AE, and AWB control [J]. IEEE Transactions on Consumer Electronics, 2001, 47(33): 694-699.
[31] Feng Q, Han K, Zhu X-C. An auto-focusing method for different object distance situation [J]. International Journal of Computer Science and Network Security, 2007, 7(6): 31-35.
[32]三星电机株式会社.自动聚焦方法及使用其的自动聚焦装置[P].中国专利: 200710079748.5, 2007-09-12.
[33]三洋电机株式会社.自动聚焦摄像机[P].中国专利: 98107054.X, 1998-11-25.
[34]盛云,秦亚杰,洪志良.一种用于数码视频相机的实时自动聚焦算法[J].光电工程, 2008, 35(8): 139-144.
[35]刘焕雨,熊文卓,万秋华.基于图像处理方法的自动调焦系统的研制[J].测试技术学报, 2007, 21(1): 13-16.
[36]李逢磊.基于FPGA的自动聚焦系统设计[D].太原理工大学, 2008年.
[37]潘广贞,陈华兵.工业摄像机自动聚焦系统设计[J].科技情报开发与经济, 2005, 15(9): 168-170.
[38]王超素,杨鸣,丁海波.一种基于视频信号处理的显微镜自动聚焦方法[J].宁波大学学报, 2008, 21(1): 11-14.
[39] Kehtarnavaz N, Oh H-J. Development and real-time implementation of a rule-based auto-focus algorithm [J]. Real-Time Imaging, 2003, 9(3): 197-203.
[40]王康泰,戴文战.一种基于Sobel算子和灰色关联度的图像边缘检测方法[J].计算机应用, 2006, 26(5): 1035-1047.
[41]张乐.自动聚焦系统精确性和实时性的研究[D].山东大学硕士论文, 2008年.
[42]高赞.自动聚焦评价函数的精确度和稳定性研究[D].山东大学硕士论文, 2007年.
[43]王欣,安志勇,杨瑞宁.基于图像清晰度评价函数的CCD摄像机自动调焦技术研究[J].长春理工大学学报, 2008, 31(1): 11-14.
[44] Chang H-C, Shih T-M, Chen N-Z. A microscope system based on bevel-axial method auto-focus [J]. Optics and Lasers in Engineering, 2009, 47(5): 547-551.
[45]袁珂,徐蔚鸿.基于图像清晰度评价的摄像头辅助调焦系统[J].光电工程, 2006, 33(1):141-144.
[46] ADI. ADSP-BF533 Blackfin Processor Hardware Reference [EB/OL]. http://www.analog.com/static/imported-files/processor_manuals/bf533_hwr_Rev3.4.pdf, 2009.
[47] ADI. Blackfin Processor Programming Reference [EB/OL]. http://www.analog.com/static/imported-files/processor_manuals/blackfin_pgr.ref.man.rev1.3.pdf, 2008.
[48] ADI. VisualDSP++ 4.5 User's Guide [EB/OL]. http://www.analog.com/static/imported-files/software_manuals_legacy/63652593555244616414545245_getstarted_man.pdf, 2006.
[49] ADI. ADSP-BF533 EZ-KIT Lite Evaluation System Manual [EB/OL]. http://www.analog.com/static/imported-files/eval_kit_manuals/288418312ADSP_BF533_EZ_KIT_Lite_Manual_Rev._3.1.pdf, 2007.
[50] ADI. VisualDSP++ 4.5 C/C++ Compiler and Library Manual for Blackfin Processors [EB/OL]. http://www.analog.com/static/imported-files/software_manuals_legacy/62641591695266839372613293690438645_Blackfin_comp_man.pdf, 2006.
[51] ADI. VisualDSP++ 4.5 Device Drivers and System Services Manual for Blackfin Processors [EB/OL]. http://www.analog.com/static/imported-files/software_manuals_legacy/6319559985497894170437445_systemservices_man_Rev21.pdf, 2006.