基于FDST和双通道PCNN的红外与可见光图像融合
|
摘要
为提高融合图像的细节表现力和信息冗余度,针对红外与可见光图像,提出一种基于有限离散剪切波变换(FDST)和双通道脉冲耦合神经网络(PCNN)的图像融合方法。首先,利用FDST分解红外与可见光图像得到各自的高低频子带系数;再对高低频子带系数分别采用不同链接强度的改进的空间频率激励的双通道PCNN进行融合;最后,通过FDST反变换得到融合图像。实验结果表明该算法能够有效增强图像清晰度和整体视觉效果,融合效果跟其他融合方法相比,在互信息、边缘信息传递量、标准差多个客观评价指标上具有明显提高。
To enhance fusion effects of infrared and visible images in detail preservation and information redundancy, a novel fusion method based on Finite Discrete Shearlet Transform(FDST) and dual-channel Pulse Coupled Neuron Network(PCNN) was proposed. Firstly, the original images were decomposed into low-frequency and high-frequency subband images by FDST; Secondly, low-frequency and high-frequency subband images were fused by modified-spatial-frequency motivated dual-channel PCNN with different linking strengths; Finally, the final fused image was reconstructed from fused subband images by inverse FDST. Experimental results indicate that the proposed fusion method can improve the overall visual performance and the image quality. Compared with other fusion methods, the proposed fusion method gets significant improvement in objective evaluation criteria of mutual information, edge information preservation and standard deviation.
To enhance fusion effects of infrared and visible images in detail preservation and information redundancy, a novel fusion method based on Finite Discrete Shearlet Transform(FDST) and dual-channel Pulse Coupled Neuron Network(PCNN) was proposed. Firstly, the original images were decomposed into low-frequency and high-frequency subband images by FDST; Secondly, low-frequency and high-frequency subband images were fused by modified-spatial-frequency motivated dual-channel PCNN with different linking strengths; Finally, the final fused image was reconstructed from fused subband images by inverse FDST. Experimental results indicate that the proposed fusion method can improve the overall visual performance and the image quality. Compared with other fusion methods, the proposed fusion method gets significant improvement in objective evaluation criteria of mutual information, edge information preservation and standard deviation.
引文
[1]Guo Quanmin,Dong Liang,Li Daiti.Vehicles antihalation system based on infrared and visible images fusion[J].Infrared and Laser Engineering,2017,46(8):0818005.(in Chinese)郭全民,董亮,李代娣.红外与可见光图像融合的汽车抗晕光系统[J].红外与激光工程,2017,46(8):0818005.
[2]Wang Xin,Ji Tongbai,Liu Fu.Fusion of infrared and visible images based on target segmentation and compressed sensing[J].Optics and Precision Engineering,2016,24(7):1743-1753.(in Chinese)王昕,吉桐伯,刘富.结合目标提取和压缩感知的红外与可见光图像融合[J].光学精密工程,2016,24(7):1743-1753.
[3]Peng Yiyue,He Weiji,Gu Guohua,et al.Visible and infrared image fusion based on multi-level method and image contrast improvement[J].Infrared and Laser Engineering,2013,42(4):1095-1099.(in Chinese)彭逸月,何伟基,顾国华,等.多层级及对比度提升的红外和可见光图像融合[J].红外与激光工程,2013,42(4):1095-1099.
[4]Hauser S,Steidl G.Convex multiclass segmentation with shearlet regularization[J].International Journal of Computer Mathematics,2013,90(1):62-81.
[5]El-taweel G S,Helmy A K.Image fusion scheme based on modified dual pulse coupled neural network[J].IETImage Processing,2013,7(5):407-414.
[6]Liu Shengpeng,Fang Yong.Infrared image fusion algorithm based on contourlet transform and improved pulse coupled neural network[J].Journal of Infrared and Millimeter Waves,2007,26(3):217-221.(in Chinese)刘盛鹏,方勇.基于Contourlet变换和IPCNN的融合算法及其在可见光与红外线图像融合中的应用[J].红外与毫米波学报,2007,26(3):217-221.
[7]Qu X B,Yan J W,Xiao H Z,et al.Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain[J].Acta Automatica Sinica,2008,34(12):1508-1514.
[8]Geng P,Wang Z Y.Zhang Z G,et al.Image fusion by pulse couple neural network with shearlet[J].Optical Engineering,2012,51(6):067005.
[9]Kong W W,Zhang L J,Lei Y.Novel fusion method for visible light and infrared images based on NSST-SF-PCNN[J].Infrared Physics&Technology,2014,65:103-112.
[10]Jiang Ping,Zhang Qiang,Li Jing,et al.Fusion algorithm for infrared and visible image based on NSSTand adaptive PCNN[J].Laser&Infrared,2014,44(1):108-113.(in Chinese)江平,张强,李静,等.基于NSST和自适应PCNN的图像融合算法[J].激光与红外,2014,44(1):108-113.
[11]Liu S Q,Shi M Z,Zhao J,et al.Multi-focus image fusion based on nonsubsampled shearlet transform and pulse coupled neural network with self-similarity and depth information[J].International Journal of Applied Mathematics and Machine Learning,2015,1(2):47-65.
[12]Xiang T Z,Yan L,Gao R R.A fusion algorithm for infrared and visible images based on adaptive dualchannel unit-linking PCNN in NSCT domain[J].Infrared Physics&Technology,2015,69:53-61.
[13]Chen Qingjiang,Zhang Yanbo,Chai Yuzhou,et al.Fusion of infrared and visible images based on finite discrete shearlet domain[J].Chinese Optics,2016,9(5):523-531.(in Chinese)陈清江,张彦博,柴昱洲,等.有限离散剪切波域的红外可见光图像融合[J].中国光学,2016,9(5):523-531.
[14]Shi Xueping,Han Xiaojun,Zhang Nan,et al.Multifocus image fusion method based on NSCT and dual PCNN[J].Microelectronics and Computer,2016,33(8):29-33.(in Chinese)石学萍,韩晓军,张南,等.基于NSCT与双通道PCNN的多聚焦图像融合[J].微电子学与计算机,2016,33(8):29-33.
[15]Qu Xiaobo,Yan Jingwen,Yang Guide.Multi-focus image fusion method of sharp frequency localized Contourlet transform domain based on sum-modifiedLaplacian[J].Optics and Precision Engineering,2009,17(5):1203-1212.(in Chinese)屈小波,闫敬文,杨贵德.改进拉普拉斯能量和的尖锐频率局部化Contourlet域多聚焦图像融合方法[J].光学精密工程,2009,17(5):1203-1212.
[16]Zhou S J,Sim T.Defocus map estimation from a single image[J].Pattern Recognition,2011,44(9):1852-1858.
[17]Toet A.The TNO multiband image data collection[J].Data in Brief,2017,15:249-251.
[2]Wang Xin,Ji Tongbai,Liu Fu.Fusion of infrared and visible images based on target segmentation and compressed sensing[J].Optics and Precision Engineering,2016,24(7):1743-1753.(in Chinese)王昕,吉桐伯,刘富.结合目标提取和压缩感知的红外与可见光图像融合[J].光学精密工程,2016,24(7):1743-1753.
[3]Peng Yiyue,He Weiji,Gu Guohua,et al.Visible and infrared image fusion based on multi-level method and image contrast improvement[J].Infrared and Laser Engineering,2013,42(4):1095-1099.(in Chinese)彭逸月,何伟基,顾国华,等.多层级及对比度提升的红外和可见光图像融合[J].红外与激光工程,2013,42(4):1095-1099.
[4]Hauser S,Steidl G.Convex multiclass segmentation with shearlet regularization[J].International Journal of Computer Mathematics,2013,90(1):62-81.
[5]El-taweel G S,Helmy A K.Image fusion scheme based on modified dual pulse coupled neural network[J].IETImage Processing,2013,7(5):407-414.
[6]Liu Shengpeng,Fang Yong.Infrared image fusion algorithm based on contourlet transform and improved pulse coupled neural network[J].Journal of Infrared and Millimeter Waves,2007,26(3):217-221.(in Chinese)刘盛鹏,方勇.基于Contourlet变换和IPCNN的融合算法及其在可见光与红外线图像融合中的应用[J].红外与毫米波学报,2007,26(3):217-221.
[7]Qu X B,Yan J W,Xiao H Z,et al.Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain[J].Acta Automatica Sinica,2008,34(12):1508-1514.
[8]Geng P,Wang Z Y.Zhang Z G,et al.Image fusion by pulse couple neural network with shearlet[J].Optical Engineering,2012,51(6):067005.
[9]Kong W W,Zhang L J,Lei Y.Novel fusion method for visible light and infrared images based on NSST-SF-PCNN[J].Infrared Physics&Technology,2014,65:103-112.
[10]Jiang Ping,Zhang Qiang,Li Jing,et al.Fusion algorithm for infrared and visible image based on NSSTand adaptive PCNN[J].Laser&Infrared,2014,44(1):108-113.(in Chinese)江平,张强,李静,等.基于NSST和自适应PCNN的图像融合算法[J].激光与红外,2014,44(1):108-113.
[11]Liu S Q,Shi M Z,Zhao J,et al.Multi-focus image fusion based on nonsubsampled shearlet transform and pulse coupled neural network with self-similarity and depth information[J].International Journal of Applied Mathematics and Machine Learning,2015,1(2):47-65.
[12]Xiang T Z,Yan L,Gao R R.A fusion algorithm for infrared and visible images based on adaptive dualchannel unit-linking PCNN in NSCT domain[J].Infrared Physics&Technology,2015,69:53-61.
[13]Chen Qingjiang,Zhang Yanbo,Chai Yuzhou,et al.Fusion of infrared and visible images based on finite discrete shearlet domain[J].Chinese Optics,2016,9(5):523-531.(in Chinese)陈清江,张彦博,柴昱洲,等.有限离散剪切波域的红外可见光图像融合[J].中国光学,2016,9(5):523-531.
[14]Shi Xueping,Han Xiaojun,Zhang Nan,et al.Multifocus image fusion method based on NSCT and dual PCNN[J].Microelectronics and Computer,2016,33(8):29-33.(in Chinese)石学萍,韩晓军,张南,等.基于NSCT与双通道PCNN的多聚焦图像融合[J].微电子学与计算机,2016,33(8):29-33.
[15]Qu Xiaobo,Yan Jingwen,Yang Guide.Multi-focus image fusion method of sharp frequency localized Contourlet transform domain based on sum-modifiedLaplacian[J].Optics and Precision Engineering,2009,17(5):1203-1212.(in Chinese)屈小波,闫敬文,杨贵德.改进拉普拉斯能量和的尖锐频率局部化Contourlet域多聚焦图像融合方法[J].光学精密工程,2009,17(5):1203-1212.
[16]Zhou S J,Sim T.Defocus map estimation from a single image[J].Pattern Recognition,2011,44(9):1852-1858.
[17]Toet A.The TNO multiband image data collection[J].Data in Brief,2017,15:249-251.