基于NSST与改进PCNN的红外与可见光图像融合方法
|
摘要
针对目标红外图像与可见光图像信息优势互补的需求,引入改进的脉冲耦合神经网络,提出一种新颖的基于非下采样剪切波变换的红外与可见光图像融合算法。首先选取非下采样剪切波变换将图像进行分解,获得高低频分量;其次低频分量的融合是利用改进空间频率作用脉冲耦合神经网络输入激励,且其链接强度由表征图像信息的平均梯度自适应调整;而高频分量处理方法是利用局部平均梯度与区域方差自适应加权融合;最后,对分别处理后的低高频分量经过非下采样剪切波变换可逆变换获取融合图像。实验结果表明,该算法可以有效综合图像的优势信息,融合结果在主观与客观评价上比经典算法更好。
Aiming at the difference characteristics between infrared image target information and visible images detail information,this paper introduce an improved pulse coupled neural network(PCNN) and propose a novel infrared and visible images fusion algorithm based on Non-subsampled Shearlet Transform(NSST).Firstly,we obtain the high and low frequency components by using the NSST multi-scale decomposition of the strictly registered source images.Secondly,the low frequency components are fused by using the modified spatial frequency as the external excitation of the PCNN,at the same time,the average gradient of low frequency components are used to adjust the link strength adaptively.Moreover,for the high frequency components,we present a self-adaptive fusion rule algorithm based on local area variance and local area average gradient.Finally,this paper uses the NSST inverse transform method to fuse low and high frequency components to obtain a fused image.Experimental results show that the proposed method of image fusion can effectively integrate important information in infrared and visible images,and the fusion effect is better than those of the general image fusion methods.
Aiming at the difference characteristics between infrared image target information and visible images detail information,this paper introduce an improved pulse coupled neural network(PCNN) and propose a novel infrared and visible images fusion algorithm based on Non-subsampled Shearlet Transform(NSST).Firstly,we obtain the high and low frequency components by using the NSST multi-scale decomposition of the strictly registered source images.Secondly,the low frequency components are fused by using the modified spatial frequency as the external excitation of the PCNN,at the same time,the average gradient of low frequency components are used to adjust the link strength adaptively.Moreover,for the high frequency components,we present a self-adaptive fusion rule algorithm based on local area variance and local area average gradient.Finally,this paper uses the NSST inverse transform method to fuse low and high frequency components to obtain a fused image.Experimental results show that the proposed method of image fusion can effectively integrate important information in infrared and visible images,and the fusion effect is better than those of the general image fusion methods.
引文
[1] HUANG Hui,ZHANG Bao-hui,XI Feng,et al.Image fusion technique based on target-enhancement[J].Infrared Technology,2017,39(10):908-913.黄慧,张宝辉,席峰,吴旭东.基于目标增强的红外与可见光图像融合技术研究[J].红外技术,2017,39(10):908-913.
[2] LIU Jian,LEI Ying-jie,XING Ya-qiong.Image fusion method based on improved NSST transform[J].Control and Decision,2017,32(2):275-280.刘健,雷英杰,邢雅琼,等.基于改进型NSST变换的图像融合方法[J].控制与决策,2017,32(2):275-280.
[3] CHEN Mu-sheng,CAI Zhi-shan.Research on fusion of infrared and visible images based on NSCT[J].Laser & Optoelectronics Progress,2015,52(6):114-119.陈木生,蔡植善.基于NSCT的红外与可见光图像融合方法研究[J].激光与光电子学进展,2015,52(6):114-119.
[4] Ge W,Ji P,Zhao T.Infrared image and visual image fusion algorithm based on NSCT and improved weight average[A].The 6th International Conference on Intelligent Systems Design and Engineering Applications[C].2016,456-459.
[5] CHEN Mu-sheng.Image fusion of visual and infrared image based on NSCT and compressed sensing[J].Journal of Image & Graphics,2016,21(1):39-44.陈木生.结合NSCT和压缩感知的红外与可见光图像融合[J].中国图象图形学报,2016,21(1):39-44.
[6] FU Zhi-zhong,WANG Xue,LI Xiao-feng,et al.Fusion of infrared and visible images based on visual significance and NSCT[J].Journal of University Of Electronic Science And Technology of China,2017,46(2):357-362.傅志中,王雪,李晓峰,等.基于视觉显著性和NSCT的红外与可见光图像融合[J].电子科技大学学报,2017,46(2):357-362.
[7] GENG Peng,WANG Zheng-you,ZHANG Zhi-gang,et a1.Image fusion by pulse couple neural network with shearlet[J].Optical Engineering,2012,51(6):1-7.
[8] Shi Z,Zhang Z,Yue Y G.Adaptive image fusion algorithm based on shearlet transform[J].Acta Photonica Sinica,2013,42(1):115-120.
[9] LIAO Yong, HUANG Wen-long, SHANG Lin,et al.Image fusion based on Shearlet and improved PCNN[J].Computer Engineering and Applications,2015,50(2):142-146.廖勇,黄文龙,尚琳,等.Shearlet与改进PCNN相结合的图像融合[J].计算机工程与应用,2015,50(2):142-146.
[10] GE Wen, JI Peng-chong, ZHAO Tian-chen. Infrared and visible image fusion based on NSST and CS[J].Laser and Infrared,2016,46(4):502-506.葛雯,姬鹏冲,赵天臣.基于NSST和CS的红外与可见光图像融合[J].激光与红外,2016,46(4):502-506.
[11] WU Dong-peng,BI Du-yan,HE lin-yuan,et al.A fusion algorithm of infrared and visible image based on NSSCT[J].Acta Optica Sinica,2017,37(7):0710003.吴冬鹏,毕笃彦,何林远,等.基于NSSCT的红外与可见光图像融合[J].光学学报,2017,37(7):0710003.
[12] Kong W,Zhang L,Lei Y.Novel fusion method for visible light and infrared images based on NSST-SF-PCNN[J].Infrared Physics & Technology,2014,65(7):103-112.
[13] WU Yi-quan,TAO Fei-xiang.Multispectral and panchromatic image fusion based on improved projected gradient NMF in NSST domain[J].Acta Optica Sinica,2015,35(4):0410005.吴一全,陶飞翔.改进投影梯度NMF的NSST域多光谱与全色图像融合[J].光学学报,2015,35(4):0410005.
[14] GUO Ming,WANG Shu-man.Image fusion based on region and directional variance weighted entropy [J].Systems Engineering & Electronics,2013,35(4):720-724.郭明,王书满.基于区域和方向方差加权信息熵的图像融合[J].系统工程与电子技术,2013,35(4):720-724.
[15] WANG Hao-peng,LIU Ze-qian,FANG Xing,et al.Method for image fusion based on adaptive pulse coupled neural network in curvelet domain[J].Journal of Optoelectronics·Laser,2016,27(4):429-436.王昊鹏,刘泽乾,方兴,等.Curvelet域自适应脉冲耦合神经网络的图像融合方法[J].光电子·激光,2016,27(4):429-436.
[16] GAO Ying,WANG A-min,ZHI Peng-fei,et al.Image fusion algorithm based on region segmentation and lifting wavelet transform[J].Journal of Northwestern Polytechnical University,2014,32(4):569-575.高颖,王阿敏,支朋飞,等.基于区域分割与提升小波变换的图像融合算法[J].西北工业大学学报,2014,32(4):569-575.
[17] HAO Ai-zhi, ZHENG Cheng. Multi-sensor image fusion based on NSCT-PCNN transform[J].Science Technology and Engineering,2014,14(1):45-48.郝爱枝,郑晟.基于NSCT-PCNN变换的多传感器图像融合[J].科学技术与工程,2014,14(1):45-48.
[18] ZHENG Wei,ZHAO Cheng-chen,HAO Dong-mei.Thyroid image fusion based on NSST and improved PCNN[J].Opto-Electronic Engineering,2016,43(10):42-48.郑伟,赵成晨,郝冬梅.NSST与改进PCNN相结合的甲状腺图像融合[J].光电工程,2016,43(10):42-48.
[2] LIU Jian,LEI Ying-jie,XING Ya-qiong.Image fusion method based on improved NSST transform[J].Control and Decision,2017,32(2):275-280.刘健,雷英杰,邢雅琼,等.基于改进型NSST变换的图像融合方法[J].控制与决策,2017,32(2):275-280.
[3] CHEN Mu-sheng,CAI Zhi-shan.Research on fusion of infrared and visible images based on NSCT[J].Laser & Optoelectronics Progress,2015,52(6):114-119.陈木生,蔡植善.基于NSCT的红外与可见光图像融合方法研究[J].激光与光电子学进展,2015,52(6):114-119.
[4] Ge W,Ji P,Zhao T.Infrared image and visual image fusion algorithm based on NSCT and improved weight average[A].The 6th International Conference on Intelligent Systems Design and Engineering Applications[C].2016,456-459.
[5] CHEN Mu-sheng.Image fusion of visual and infrared image based on NSCT and compressed sensing[J].Journal of Image & Graphics,2016,21(1):39-44.陈木生.结合NSCT和压缩感知的红外与可见光图像融合[J].中国图象图形学报,2016,21(1):39-44.
[6] FU Zhi-zhong,WANG Xue,LI Xiao-feng,et al.Fusion of infrared and visible images based on visual significance and NSCT[J].Journal of University Of Electronic Science And Technology of China,2017,46(2):357-362.傅志中,王雪,李晓峰,等.基于视觉显著性和NSCT的红外与可见光图像融合[J].电子科技大学学报,2017,46(2):357-362.
[7] GENG Peng,WANG Zheng-you,ZHANG Zhi-gang,et a1.Image fusion by pulse couple neural network with shearlet[J].Optical Engineering,2012,51(6):1-7.
[8] Shi Z,Zhang Z,Yue Y G.Adaptive image fusion algorithm based on shearlet transform[J].Acta Photonica Sinica,2013,42(1):115-120.
[9] LIAO Yong, HUANG Wen-long, SHANG Lin,et al.Image fusion based on Shearlet and improved PCNN[J].Computer Engineering and Applications,2015,50(2):142-146.廖勇,黄文龙,尚琳,等.Shearlet与改进PCNN相结合的图像融合[J].计算机工程与应用,2015,50(2):142-146.
[10] GE Wen, JI Peng-chong, ZHAO Tian-chen. Infrared and visible image fusion based on NSST and CS[J].Laser and Infrared,2016,46(4):502-506.葛雯,姬鹏冲,赵天臣.基于NSST和CS的红外与可见光图像融合[J].激光与红外,2016,46(4):502-506.
[11] WU Dong-peng,BI Du-yan,HE lin-yuan,et al.A fusion algorithm of infrared and visible image based on NSSCT[J].Acta Optica Sinica,2017,37(7):0710003.吴冬鹏,毕笃彦,何林远,等.基于NSSCT的红外与可见光图像融合[J].光学学报,2017,37(7):0710003.
[12] Kong W,Zhang L,Lei Y.Novel fusion method for visible light and infrared images based on NSST-SF-PCNN[J].Infrared Physics & Technology,2014,65(7):103-112.
[13] WU Yi-quan,TAO Fei-xiang.Multispectral and panchromatic image fusion based on improved projected gradient NMF in NSST domain[J].Acta Optica Sinica,2015,35(4):0410005.吴一全,陶飞翔.改进投影梯度NMF的NSST域多光谱与全色图像融合[J].光学学报,2015,35(4):0410005.
[14] GUO Ming,WANG Shu-man.Image fusion based on region and directional variance weighted entropy [J].Systems Engineering & Electronics,2013,35(4):720-724.郭明,王书满.基于区域和方向方差加权信息熵的图像融合[J].系统工程与电子技术,2013,35(4):720-724.
[15] WANG Hao-peng,LIU Ze-qian,FANG Xing,et al.Method for image fusion based on adaptive pulse coupled neural network in curvelet domain[J].Journal of Optoelectronics·Laser,2016,27(4):429-436.王昊鹏,刘泽乾,方兴,等.Curvelet域自适应脉冲耦合神经网络的图像融合方法[J].光电子·激光,2016,27(4):429-436.
[16] GAO Ying,WANG A-min,ZHI Peng-fei,et al.Image fusion algorithm based on region segmentation and lifting wavelet transform[J].Journal of Northwestern Polytechnical University,2014,32(4):569-575.高颖,王阿敏,支朋飞,等.基于区域分割与提升小波变换的图像融合算法[J].西北工业大学学报,2014,32(4):569-575.
[17] HAO Ai-zhi, ZHENG Cheng. Multi-sensor image fusion based on NSCT-PCNN transform[J].Science Technology and Engineering,2014,14(1):45-48.郝爱枝,郑晟.基于NSCT-PCNN变换的多传感器图像融合[J].科学技术与工程,2014,14(1):45-48.
[18] ZHENG Wei,ZHAO Cheng-chen,HAO Dong-mei.Thyroid image fusion based on NSST and improved PCNN[J].Opto-Electronic Engineering,2016,43(10):42-48.郑伟,赵成晨,郝冬梅.NSST与改进PCNN相结合的甲状腺图像融合[J].光电工程,2016,43(10):42-48.