基于超分辨率和组稀疏表示的多聚焦图像融合
|
摘要
提出一种基于超分辨率结合组稀疏表示模型的多聚焦图像融合方法.首先,使用双三次插值方法增强源图像的分辨率及源多聚焦图像信息;然后采用自适应稀疏表示学习字典分别对没有明显主导方向和特定主导方向的图像块进行学习,并采用组稀疏表示模型对源多聚焦图像进行稀疏系数表示;最后采用最大l1范数来选择最终的表示系数向量.实验结果表明,所提方法克服了多聚焦图像融合易出现的低空间分辨率和模糊效果的缺点,具有更好的对比度和清晰度,主观视觉效果和客观指标均优于传统多聚焦图像融合方法,在三组图像融合结果的互信息指标上分别领先0.37、0.38和0.32.
A multi-focus image fusion method based on super-resolution combined with group sparse representation model is proposed.First,the bicubic interpolation method is used to enhance the resolution of the source image and the source multi-focus image information.Then,the adaptive sparse representation learning dictionary is used to learn the image blocks without obvious dominant direction and specific dominant direction respectively.The sparse coefficient representation of the source multifocus image is conducted by the group sparse representation model.Finally,the maximuml1 norm is used to select the final representation coefficient vector.The experimental results show that the proposed method restrains the shortcomings of low spatial resolution and blurring that are easy to appear in multifocus image fusion,and has better contrast and sharpness.Subjective visual effects and objective indicators show that the proposed method has certain advantages over traditional multi-focus image fusion methods,especially in the mutual information index of the three sets of image fusion results leading 0.37,0.38 and 0.32 respectively.
A multi-focus image fusion method based on super-resolution combined with group sparse representation model is proposed.First,the bicubic interpolation method is used to enhance the resolution of the source image and the source multi-focus image information.Then,the adaptive sparse representation learning dictionary is used to learn the image blocks without obvious dominant direction and specific dominant direction respectively.The sparse coefficient representation of the source multifocus image is conducted by the group sparse representation model.Finally,the maximuml1 norm is used to select the final representation coefficient vector.The experimental results show that the proposed method restrains the shortcomings of low spatial resolution and blurring that are easy to appear in multifocus image fusion,and has better contrast and sharpness.Subjective visual effects and objective indicators show that the proposed method has certain advantages over traditional multi-focus image fusion methods,especially in the mutual information index of the three sets of image fusion results leading 0.37,0.38 and 0.32 respectively.
引文
[1]FENG Xin.Fusion of infrared and visible images based on Tetrolet framework[J].Acta Photonica Sinica,2019,48(2):0210001.冯鑫.Tetrolet框架下红外与可见光图像融合[J].光子学报,2019,48(2):0210001.
[2]LI Shu-tao,KANG Xu-dong,FANG Le-yuan.Pixel-level image fusion:a survey of the state of the art[J].Information Fusion,2017,33(1):100-112.
[3]GUO Li-qiang,DAI Ming,ZHU Ming.Multifocus color image fusion based on quaternion curvelet transform[J].Optics Express,2012,20(17):18846-18860.
[4]WANG Lei,LI Bing,TIAN Lian-fang.Multi-modal medical image fusion using the inter-scale and intra-scale dependencies between image shift-invariant shearlet coefficients[J].Information Fusion,2014,19(1):20-28.
[5]UPLA K,GAJJAR P,JOSHI M.An edge preserving multiresolution fusion:use of contourlet transform and MRF prior[J].IEEE Transactions on Geoscience and Remote Sensing,2015,53(6):3210-3220.
[6]LI Shu-tao,YANG Bing,HU Jian-wen.Performance comparison of different multi-resolution transforms for image fusion[J].Information Fusion,2014,12(2):74-84.
[7]WRIGHT J,GANESH A,ZHOU Zi-han.Robust face recognition via sparse representation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,31(2):210-227.
[8]GUHA T,WARD R K.Learning sparse representations for human action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(8):1576-1588.
[9]YUAN Xiao-tong,YAN Shui-cheng.Visual classification with multitask joint sparse representation[J].IEEETransactions on Image Processing,2012,21(10):4349-4360.
[10]YANG Bing,LI Shu-tao.Multifocus image fusion and restoration with sparse representation[J].IEEE Transactions on Instrumentation and Measurement,2010,59(4):884-892.
[11]LIU Yu,LIU Shu-ping,WANG Zeng-fu.A general framework for image fusion based on multiscale transform and sparse representation[J].Information Fusion,2015,24(7):147-164.
[12]YIN Ming,ZHAN Yin-wei,PEI Hai-long.Co-sparse analysis operator learning for image fusion[J].Journal of Jilin University(Engineering and Technology Edition),2016,46(6):2052-2058.尹明,战荫伟,裴海龙.基于稀疏补算子学习的图像融合方法[J].吉林大学学报(工学版),2016,46(6):2052-2058.
[13]LIU Zhe,XU Tao,SONG Yu-qing.Image fusion technology based on NSCT and robust principal component analysis model with similar information[J].Journal of Jilin University(Engineering and Technology Edition),2018,48(5):1614-1620.刘哲,徐涛,宋余庆.基于NSCT变换和相似信息鲁棒主成分分析模型的图像融合技术[J].吉林大学学报(工学版),2018,48(5):1614-1620.
[14]ZHAO Chun-hui,GUO Yun-ting.Fast image fusion algorithm based on sparse representation and non subsampled contourlet transform[J].Journal of Electronics&Information Technology,2016,38(7):1773-1780.赵春晖,郭蕴霆.一种快速的基于稀疏表示和非下采样轮廓波变换的图像融合算法[J].电子与信息学报,2016,38(7):1773-1780.
[15]NEJATI M,SAMAVI S,SHIRANI S.Multi-focus image fusion using dictionary based sparse representation[J].Information Fusion,2015,25(9):72-84.
[16]YIN Hong-peng,LI Yan-xia,CHAI Yi.A novel sparse-representation-based multi-focus image fusion approach[J].Neurocomputing,2016,216(12):216-229.
[17]ZHANG Qiang,SHI Tao,WANG Fan.Robust sparse representation based multi-focus image fusion with dictionary construction and local spatial consistency[J].Pattern Recognition,2018,83(11):299-313.
[18]ZHANG Qiang,LIU Yi,BLUM R S,et al.Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images:a review[J].Information Fusion,2018,40(3):57-75.
[19]VIJITHA B,REDDY K S.Image reconstruction with super-resolution[J].International Journal of Research in Computer Applications and Robotics.2016,4(9):36-40.
[20]LIU Yu,WANG Zeng-fu.Simultaneous image fusion and denoising with adaptive sparse representation[J].IET Image Process,2015,9(5):347-357.
[21]LI Shu-tao,YIN Hai-tao,FANG Le-yuan.Group-sparse representation with dictionary learning for medical image denoising and fusion[J].IEEETransactions on Bio-medical Engineering,2012,59(12):3450-3459.
[22]MAJUMDAR A,WARD R K.Fast group sparse classification[J].Canadian Journal of Electrical and Computer Engineering,2016,34(4):136-144.
[23]YIN Hong-peng,LI Yan-xia,CHAI Yi,et al.A novel sparse-representation-based multi-focus image fusion approach[J].Neurocomputing,2016,216(7):16-229.
[24]LIU Zheng,ERIK B,XUE Zhi-yun.Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision:acomparative study[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(1):94-109.
[2]LI Shu-tao,KANG Xu-dong,FANG Le-yuan.Pixel-level image fusion:a survey of the state of the art[J].Information Fusion,2017,33(1):100-112.
[3]GUO Li-qiang,DAI Ming,ZHU Ming.Multifocus color image fusion based on quaternion curvelet transform[J].Optics Express,2012,20(17):18846-18860.
[4]WANG Lei,LI Bing,TIAN Lian-fang.Multi-modal medical image fusion using the inter-scale and intra-scale dependencies between image shift-invariant shearlet coefficients[J].Information Fusion,2014,19(1):20-28.
[5]UPLA K,GAJJAR P,JOSHI M.An edge preserving multiresolution fusion:use of contourlet transform and MRF prior[J].IEEE Transactions on Geoscience and Remote Sensing,2015,53(6):3210-3220.
[6]LI Shu-tao,YANG Bing,HU Jian-wen.Performance comparison of different multi-resolution transforms for image fusion[J].Information Fusion,2014,12(2):74-84.
[7]WRIGHT J,GANESH A,ZHOU Zi-han.Robust face recognition via sparse representation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,31(2):210-227.
[8]GUHA T,WARD R K.Learning sparse representations for human action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(8):1576-1588.
[9]YUAN Xiao-tong,YAN Shui-cheng.Visual classification with multitask joint sparse representation[J].IEEETransactions on Image Processing,2012,21(10):4349-4360.
[10]YANG Bing,LI Shu-tao.Multifocus image fusion and restoration with sparse representation[J].IEEE Transactions on Instrumentation and Measurement,2010,59(4):884-892.
[11]LIU Yu,LIU Shu-ping,WANG Zeng-fu.A general framework for image fusion based on multiscale transform and sparse representation[J].Information Fusion,2015,24(7):147-164.
[12]YIN Ming,ZHAN Yin-wei,PEI Hai-long.Co-sparse analysis operator learning for image fusion[J].Journal of Jilin University(Engineering and Technology Edition),2016,46(6):2052-2058.尹明,战荫伟,裴海龙.基于稀疏补算子学习的图像融合方法[J].吉林大学学报(工学版),2016,46(6):2052-2058.
[13]LIU Zhe,XU Tao,SONG Yu-qing.Image fusion technology based on NSCT and robust principal component analysis model with similar information[J].Journal of Jilin University(Engineering and Technology Edition),2018,48(5):1614-1620.刘哲,徐涛,宋余庆.基于NSCT变换和相似信息鲁棒主成分分析模型的图像融合技术[J].吉林大学学报(工学版),2018,48(5):1614-1620.
[14]ZHAO Chun-hui,GUO Yun-ting.Fast image fusion algorithm based on sparse representation and non subsampled contourlet transform[J].Journal of Electronics&Information Technology,2016,38(7):1773-1780.赵春晖,郭蕴霆.一种快速的基于稀疏表示和非下采样轮廓波变换的图像融合算法[J].电子与信息学报,2016,38(7):1773-1780.
[15]NEJATI M,SAMAVI S,SHIRANI S.Multi-focus image fusion using dictionary based sparse representation[J].Information Fusion,2015,25(9):72-84.
[16]YIN Hong-peng,LI Yan-xia,CHAI Yi.A novel sparse-representation-based multi-focus image fusion approach[J].Neurocomputing,2016,216(12):216-229.
[17]ZHANG Qiang,SHI Tao,WANG Fan.Robust sparse representation based multi-focus image fusion with dictionary construction and local spatial consistency[J].Pattern Recognition,2018,83(11):299-313.
[18]ZHANG Qiang,LIU Yi,BLUM R S,et al.Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images:a review[J].Information Fusion,2018,40(3):57-75.
[19]VIJITHA B,REDDY K S.Image reconstruction with super-resolution[J].International Journal of Research in Computer Applications and Robotics.2016,4(9):36-40.
[20]LIU Yu,WANG Zeng-fu.Simultaneous image fusion and denoising with adaptive sparse representation[J].IET Image Process,2015,9(5):347-357.
[21]LI Shu-tao,YIN Hai-tao,FANG Le-yuan.Group-sparse representation with dictionary learning for medical image denoising and fusion[J].IEEETransactions on Bio-medical Engineering,2012,59(12):3450-3459.
[22]MAJUMDAR A,WARD R K.Fast group sparse classification[J].Canadian Journal of Electrical and Computer Engineering,2016,34(4):136-144.
[23]YIN Hong-peng,LI Yan-xia,CHAI Yi,et al.A novel sparse-representation-based multi-focus image fusion approach[J].Neurocomputing,2016,216(7):16-229.
[24]LIU Zheng,ERIK B,XUE Zhi-yun.Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision:acomparative study[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(1):94-109.