基于FPGA的大动态范围成像反馈系统研究
摘要
图像信息技术是信息领域包含信息量最大最重要的分支之一,在许多重要的应用领域,固体图像传感器的动态范围起着至关重要的作用,所以扩展固体传感器的动态范围是获取更多图像信息的重要手段之一。
本课题研究一种大动态范围的成像(HDRI,Hign Dynamic Range Image)技术,它采用空间光调制器和固体图像传感器相结合,根据固体图像传感器的输出信号调节空间光调制器的调制深度,组成一个负反馈闭环控制系统,达到扩展固体成像器件动态范围的目的。论文的主要工作有:
(1)在总结分析了现有扩展固体图像传感器动态范围扩展的基础上,对系统的反馈过程进行了分析及仿真,当在一定的反馈深度范围内时,确认采用直接反馈控制的可行性,为脱机HDRI系统方案提供了理论依据以及可行性。
(2)根据HDRI系统需求,比较多种方案的基础上,提出了基于FPGA为核心的HDRI系统。
(3)结合FPGA和HDRI系统,完善了整个系统的外围电路及功能模块规划,完成了整个方案的器件选型、电路设计、加工以及调试。
(4)初步完成了HDRI系统各功能单元的测试,完成了基于FPGA的图像采集、数据传输、数据缓冲以及标准的XGA格式的视频差分信号输出。实验结果表明,各功能模块能满足HDRI系统的要求,达到了预期的效果。
Image information is one of the information technology branches which include the most part of information field. In many important applications, the dynamic range of solid-state imaging sensors is critical. As a result, the extension of dynamic range of solid-state imaging sensors is one of the key methods to extract more information from imaging.
This project developed a high dynamic range imaging (HDRI) system based on spatial light modulator and solid-state imaging sensors. The depth of modulation of spatial light modulator is determined by the output of solid-state imaging sensors. Hence, a negative-loop control system is constituted to extend the dynamic range of solid-state sensors. This thesis includes the following parts:
(1)Based on summing up and analyzing expansion dynamic range of the existing solid image sensor, the feedback control is analyzed and formulated. The feasibility of direct feedback control is studied with simulation. The range of feedback depth is discovered for the convergence of output images.
(2) According to the requirement of HDRI system, FPGA is selected as a main-controller based on comparing with schemes.
(3) According to HDRI and FPGA, peripheral circuit is drawn and module is laidout.Chip selecting and circuit designs are accomplished.
(4) Function module of HDRI system is debugged elementary .And data acquired,processed and transferred in ping-pong mode based on FPGA is achieved.Experiment indicates that function module of HDRI system meet the demand of HDRI system and forecast resulets are achieve.
本课题研究一种大动态范围的成像(HDRI,Hign Dynamic Range Image)技术,它采用空间光调制器和固体图像传感器相结合,根据固体图像传感器的输出信号调节空间光调制器的调制深度,组成一个负反馈闭环控制系统,达到扩展固体成像器件动态范围的目的。论文的主要工作有:
(1)在总结分析了现有扩展固体图像传感器动态范围扩展的基础上,对系统的反馈过程进行了分析及仿真,当在一定的反馈深度范围内时,确认采用直接反馈控制的可行性,为脱机HDRI系统方案提供了理论依据以及可行性。
(2)根据HDRI系统需求,比较多种方案的基础上,提出了基于FPGA为核心的HDRI系统。
(3)结合FPGA和HDRI系统,完善了整个系统的外围电路及功能模块规划,完成了整个方案的器件选型、电路设计、加工以及调试。
(4)初步完成了HDRI系统各功能单元的测试,完成了基于FPGA的图像采集、数据传输、数据缓冲以及标准的XGA格式的视频差分信号输出。实验结果表明,各功能模块能满足HDRI系统的要求,达到了预期的效果。
Image information is one of the information technology branches which include the most part of information field. In many important applications, the dynamic range of solid-state imaging sensors is critical. As a result, the extension of dynamic range of solid-state imaging sensors is one of the key methods to extract more information from imaging.
This project developed a high dynamic range imaging (HDRI) system based on spatial light modulator and solid-state imaging sensors. The depth of modulation of spatial light modulator is determined by the output of solid-state imaging sensors. Hence, a negative-loop control system is constituted to extend the dynamic range of solid-state sensors. This thesis includes the following parts:
(1)Based on summing up and analyzing expansion dynamic range of the existing solid image sensor, the feedback control is analyzed and formulated. The feasibility of direct feedback control is studied with simulation. The range of feedback depth is discovered for the convergence of output images.
(2) According to the requirement of HDRI system, FPGA is selected as a main-controller based on comparing with schemes.
(3) According to HDRI and FPGA, peripheral circuit is drawn and module is laidout.Chip selecting and circuit designs are accomplished.
(4) Function module of HDRI system is debugged elementary .And data acquired,processed and transferred in ping-pong mode based on FPGA is achieved.Experiment indicates that function module of HDRI system meet the demand of HDRI system and forecast resulets are achieve.
引文
[1].唐晓华,陈君良.沈括与《梦溪笔谈》[J].物理教学探讨,2005,23(2):39-39.
[2].陈申.北京早期照相馆史料概况[J].文史纵横,2005,2:38-41.
[3].王庆友,图像传感器技术[M],北京:电子工业出版社,2003,1-2.
[4].曾德贤,赵继广.CMOS图像传感器的三种新技术[J].光机电信息,2004,11:24-27.
[5].裴志军,国澄明,姚素英,等.CMOS图像传感器动态范围扩展技术[J].传感器技术,2003,22(6):1-4
[6].William K.Pratt.Machine vision methods for automatic defect detection in liquid crystal displays[J].Advanced Imaging,1998 13(4):53-54.
[7].Clogg Phil,Diaz—Anderu Margarita,Larkman Brian.Digital image processing and the recording of rock art[J].Archaeological Science,2000,7(9):837-843.
[8].David X.D.Yang,Abbas EI Gamal,et al.,A 640X512 CMOS Image Sensor with UltrawideDynamic RangeFloating-Point Pixel-Level ADC[J],IEEE Journal of Solid-State Circuits,1999,34(12):1821-1834.
[9].Abbas El Gamal,SNR and Dynamic Range[J],EE392B Lecture Note 5,Stanford University,Stanford,USA,2001
[10].Yasuaki Nishida,Junro Koike,Toshihide Watanabe.Wide Dynamic Range HDTV Image Sensor with Aliasing Suppression[J],IEEE TeE,1988,34(3):506-511.
[11].M.Sayag,Nonlinear photosite response in CCD imagers,1991,U.S.Patent 5055667
[12].谈新权,梅晓英.高分辨率CCD图像传感器及CCD摄像机的性能评价[J].敏通科技1999(12):20-22
[13].Otim S.O.,Joseph D.,Choubey B.,et al.,Modeling of high dynamic range logarithmic CMOS image sensors[J].Proceedings of the 21st IEEE Instrumentation and Measurement Technology Conference,2004,1:451-456.
[14].徐江涛,姚素英,赵毅强等,新型大动态范围CMOS图像传感器双采样像素阵列设计[J].传感技术学报,2003,16(4):438-441.
[15].Kubo,N.,Takemura,K.,Adachi,K,et.al.,Super-dynamic-range image processing system using a new structure CCD[J].SPIE,2004,5301:303-310.
[16].http://www.fujifilm.com.cn/en/ena-tech2.htm,2001-07-19
[17].Robertson M.A.,Borman S.,Stevenson R.L.,Estimation-theoretic approach to dynamic range enhancement using multiple exposures[J].Journal of Electronic Imaging,2003,12(2):219-228
[18].L.BOGONI,Extending Dynamic Range of Monochrome and Color Images through Fusion [C],Proceedings of the 15th International Conference on Pattern Recognition,Barcelona,Spain:IEEE Computer Society,2000,3:7-12.
[19].何烽,徐之海,冯华君,等.一种基于数字图像合成的扩展动态范围方法[J].光电工程,2003,30(5),66-68.
[20].Mannos J.L,Sakrison D.J.The Effect of a Visual Fidelity Criterion on the Encoding of Images[J]. IEEE Trans.Inform Theory,1974,20:525-536
[21].Shree K Nayar and Vlad Branzoi,Programmable Imaging using a Digital Micromirror Array[J],Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR'04),1063-6919/04:436-443
[22].徐进,陈浙泊,倪旭翔,陆祖康,基于PID控制器的HDRI系统研究[J],光学技术,2008,4:51-53
[23].Jill Xu,Zhebo Chen,Xuxiang Ni,Zukang Lu,The stability research of one kind of high dynamic range imaging system[J],Optical Eugneering,2008,47(3):033202-1-4.
[24].徐志军,徐光辉.CPLD/FPGA的开发与应用[M].北京:电子工业出版社,2002:3-9
[25].张宇天,等.几款视频处理DSP性能及应用对比[J].中国公共安全,2004,10:122-124
[26].任爱锋,等.基于FPGA的嵌入式系统设计[M],西安:西安电子科技大学出版社,2004:13-17
[27].金明,罗飞路,等.FIFO芯片在高速系统中的应用[J],电子应用技术,1998,24(3):61-62
[28].吴国新.构成大容量非易失性SRAM方法分析[J].电测与仪表,2002,39(2):51-53
[29].Databook.2MX32 SDRAM 512kX32bitX4Banks Synchronous DRAM LVTTL,Hynix Inc.,2002
[30].OVT CameraChip OV9620/9120,OmniVision Technologies,Inc.2001
[31].Infocus Corp,Digital Visual Interface White Paper[EB/OL],http://www.infocus.com,November 2001
[32].SiI 160 PanelLink Transmitter Data Sher,Silicon Image,Inc.August 2002
[33].Cyclone,Ⅱ Device Handbook,Volume 1,AItera Corporation,August 2007
[34].徐光辉,程东旭,等.基于FPGA的嵌入式开发与应用,北京:电子工业出版社,2006,9:24-50
[35].EDA先锋工作室.AlteraFPGA/CPLD设计(基础篇),北京:人民邮电出版社,2005,7:1-18
[36].Quartus Ⅱ 简介.www.altera.com,2005,4
[37].Michael D.Ciletti,Advanced Digital Design with the Verilog HDL[M],Beijing,Publishing House of Electronics Industry,2004.4:50-78
[38].Datasheet.Synchronous DRAM MT48LC2M32B2-512Kx32x4 bankd,Micron Technology,Inc.2001,10
[39].SDR SDRAM Controller Application Note AN8064,Lattice Semiconductor Corporation,July 2001
[40].Shuguang Wu,Mohammed Farhat,Graphics Engineers,et.al.,Proposed Display Timing Standard Addresses Demands of Digital Flat-Panel Displays,www.dell.com/r&d,January 2003
[41].EDA先锋工作室.AlteraFPGA/CPLD设计(高级篇),北京:人民邮电出版社,2005,7:1-100
[42].VijayA.Nebhrajani,Asynchronous FIFO Architectures,Cisco Systems,San jose,2005,7:1-18
[43].Altera中国区授权代理骏龙科技有限公司(技术支持部),使用Cyclone器件中的PLL,www.Cytech.com,
[44].夏宇闻.Verilog数字系统设计教程,北京:北京航空航天大学出版社,2003,7:369-374
[45]．徐欣,孙广福．HDL编码风格与编码指南,www.IPcore.com.cn, June 30,2002,: 1-17
[46]. SDRAM DEVICE OPERATION, Hynix,Inc, 2003, 9: 1-15
[47]. Vijay A. Nebhrajani, Asynchronous FIFO Architectures, Cisco Systems, San jose, 2005, 7: 1 —18
[48]. H. Ying. Practical Design of Nonlinear Fuzzy Controllers With Stability Analysis for Regulating Processes with Unknown Mathematical Models [J].Automatica, 1999,66:12-15
[2].陈申.北京早期照相馆史料概况[J].文史纵横,2005,2:38-41.
[3].王庆友,图像传感器技术[M],北京:电子工业出版社,2003,1-2.
[4].曾德贤,赵继广.CMOS图像传感器的三种新技术[J].光机电信息,2004,11:24-27.
[5].裴志军,国澄明,姚素英,等.CMOS图像传感器动态范围扩展技术[J].传感器技术,2003,22(6):1-4
[6].William K.Pratt.Machine vision methods for automatic defect detection in liquid crystal displays[J].Advanced Imaging,1998 13(4):53-54.
[7].Clogg Phil,Diaz—Anderu Margarita,Larkman Brian.Digital image processing and the recording of rock art[J].Archaeological Science,2000,7(9):837-843.
[8].David X.D.Yang,Abbas EI Gamal,et al.,A 640X512 CMOS Image Sensor with UltrawideDynamic RangeFloating-Point Pixel-Level ADC[J],IEEE Journal of Solid-State Circuits,1999,34(12):1821-1834.
[9].Abbas El Gamal,SNR and Dynamic Range[J],EE392B Lecture Note 5,Stanford University,Stanford,USA,2001
[10].Yasuaki Nishida,Junro Koike,Toshihide Watanabe.Wide Dynamic Range HDTV Image Sensor with Aliasing Suppression[J],IEEE TeE,1988,34(3):506-511.
[11].M.Sayag,Nonlinear photosite response in CCD imagers,1991,U.S.Patent 5055667
[12].谈新权,梅晓英.高分辨率CCD图像传感器及CCD摄像机的性能评价[J].敏通科技1999(12):20-22
[13].Otim S.O.,Joseph D.,Choubey B.,et al.,Modeling of high dynamic range logarithmic CMOS image sensors[J].Proceedings of the 21st IEEE Instrumentation and Measurement Technology Conference,2004,1:451-456.
[14].徐江涛,姚素英,赵毅强等,新型大动态范围CMOS图像传感器双采样像素阵列设计[J].传感技术学报,2003,16(4):438-441.
[15].Kubo,N.,Takemura,K.,Adachi,K,et.al.,Super-dynamic-range image processing system using a new structure CCD[J].SPIE,2004,5301:303-310.
[16].http://www.fujifilm.com.cn/en/ena-tech2.htm,2001-07-19
[17].Robertson M.A.,Borman S.,Stevenson R.L.,Estimation-theoretic approach to dynamic range enhancement using multiple exposures[J].Journal of Electronic Imaging,2003,12(2):219-228
[18].L.BOGONI,Extending Dynamic Range of Monochrome and Color Images through Fusion [C],Proceedings of the 15th International Conference on Pattern Recognition,Barcelona,Spain:IEEE Computer Society,2000,3:7-12.
[19].何烽,徐之海,冯华君,等.一种基于数字图像合成的扩展动态范围方法[J].光电工程,2003,30(5),66-68.
[20].Mannos J.L,Sakrison D.J.The Effect of a Visual Fidelity Criterion on the Encoding of Images[J]. IEEE Trans.Inform Theory,1974,20:525-536
[21].Shree K Nayar and Vlad Branzoi,Programmable Imaging using a Digital Micromirror Array[J],Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR'04),1063-6919/04:436-443
[22].徐进,陈浙泊,倪旭翔,陆祖康,基于PID控制器的HDRI系统研究[J],光学技术,2008,4:51-53
[23].Jill Xu,Zhebo Chen,Xuxiang Ni,Zukang Lu,The stability research of one kind of high dynamic range imaging system[J],Optical Eugneering,2008,47(3):033202-1-4.
[24].徐志军,徐光辉.CPLD/FPGA的开发与应用[M].北京:电子工业出版社,2002:3-9
[25].张宇天,等.几款视频处理DSP性能及应用对比[J].中国公共安全,2004,10:122-124
[26].任爱锋,等.基于FPGA的嵌入式系统设计[M],西安:西安电子科技大学出版社,2004:13-17
[27].金明,罗飞路,等.FIFO芯片在高速系统中的应用[J],电子应用技术,1998,24(3):61-62
[28].吴国新.构成大容量非易失性SRAM方法分析[J].电测与仪表,2002,39(2):51-53
[29].Databook.2MX32 SDRAM 512kX32bitX4Banks Synchronous DRAM LVTTL,Hynix Inc.,2002
[30].OVT CameraChip OV9620/9120,OmniVision Technologies,Inc.2001
[31].Infocus Corp,Digital Visual Interface White Paper[EB/OL],http://www.infocus.com,November 2001
[32].SiI 160 PanelLink Transmitter Data Sher,Silicon Image,Inc.August 2002
[33].Cyclone,Ⅱ Device Handbook,Volume 1,AItera Corporation,August 2007
[34].徐光辉,程东旭,等.基于FPGA的嵌入式开发与应用,北京:电子工业出版社,2006,9:24-50
[35].EDA先锋工作室.AlteraFPGA/CPLD设计(基础篇),北京:人民邮电出版社,2005,7:1-18
[36].Quartus Ⅱ 简介.www.altera.com,2005,4
[37].Michael D.Ciletti,Advanced Digital Design with the Verilog HDL[M],Beijing,Publishing House of Electronics Industry,2004.4:50-78
[38].Datasheet.Synchronous DRAM MT48LC2M32B2-512Kx32x4 bankd,Micron Technology,Inc.2001,10
[39].SDR SDRAM Controller Application Note AN8064,Lattice Semiconductor Corporation,July 2001
[40].Shuguang Wu,Mohammed Farhat,Graphics Engineers,et.al.,Proposed Display Timing Standard Addresses Demands of Digital Flat-Panel Displays,www.dell.com/r&d,January 2003
[41].EDA先锋工作室.AlteraFPGA/CPLD设计(高级篇),北京:人民邮电出版社,2005,7:1-100
[42].VijayA.Nebhrajani,Asynchronous FIFO Architectures,Cisco Systems,San jose,2005,7:1-18
[43].Altera中国区授权代理骏龙科技有限公司(技术支持部),使用Cyclone器件中的PLL,www.Cytech.com,
[44].夏宇闻.Verilog数字系统设计教程,北京:北京航空航天大学出版社,2003,7:369-374
[45]．徐欣,孙广福．HDL编码风格与编码指南,www.IPcore.com.cn, June 30,2002,: 1-17
[46]. SDRAM DEVICE OPERATION, Hynix,Inc, 2003, 9: 1-15
[47]. Vijay A. Nebhrajani, Asynchronous FIFO Architectures, Cisco Systems, San jose, 2005, 7: 1 —18
[48]. H. Ying. Practical Design of Nonlinear Fuzzy Controllers With Stability Analysis for Regulating Processes with Unknown Mathematical Models [J].Automatica, 1999,66:12-15