深度协作表达的人脸超分辨重建算法
|
摘要
为了获得更高质量的高分辨率人脸图像,提出一种基于深度协作表达的人脸超分辨率算法.该算法首先对训练样本重叠取块和多方向梯度特征提取得到初始化字典;再对初始化字典进行逐层迭代更新,同时利用协作表达更新对应每层的最优表达权重系数;最后将最后一个表达层所有的重建块合成为最终高分辨率图像.在FEI和CMU FrontalFace数据集上的实验结果表明,与传统超分辨率重建算法相比,该算法提升了单层表达算法的精度,在主观和客观评价性能上均超过现有算法,甚至包括基于深度学习算法.
In order to obtain high-resolution face images with better quality, a face super-resolution algorithm based on deep collaborative representation is proposed. Firstly, the algorithm extracts overlapping patches and multi-directional gradient feature to obtain the initialization dictionary. Then, the initialization dictionary is updated iteratively layer by layer, and the optimal expression weight coefficients corresponding to each layer are updated by the collaborative representation. Finally, all the reconstructed patches of the last layer are combined into a final high-resolution image. Compared with traditional super-resolution reconstruction algorithms, the experimental results on FEI and CMU Frontal Face datasets show that the proposed algorithm improves the accuracy of the single-level representation algorithm and outperforms the existing algorithms in both subjective and objective evaluation performance, even including the deep-based learning algorithm.
In order to obtain high-resolution face images with better quality, a face super-resolution algorithm based on deep collaborative representation is proposed. Firstly, the algorithm extracts overlapping patches and multi-directional gradient feature to obtain the initialization dictionary. Then, the initialization dictionary is updated iteratively layer by layer, and the optimal expression weight coefficients corresponding to each layer are updated by the collaborative representation. Finally, all the reconstructed patches of the last layer are combined into a final high-resolution image. Compared with traditional super-resolution reconstruction algorithms, the experimental results on FEI and CMU Frontal Face datasets show that the proposed algorithm improves the accuracy of the single-level representation algorithm and outperforms the existing algorithms in both subjective and objective evaluation performance, even including the deep-based learning algorithm.
引文
[1]Baker S,Kanade T.Hallucinating faces[C]//Proceedings of the4th IEEE International Conference on Automatic Face and Gesture Recognition.Los Alamitos:IEEE Computer Society Press,2000:83-88
[2]Baker S,Kanade T.Limits on super-resolution and how to break them[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2002,24(9):1167-1183
[3]Chang H,Yeung D Y,Xiong Y M.Super-resolution through neighbor embedding[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2004,1:275-282
[4]Ma X,Zhang J P,Qi C.Position-based face hallucination method[C]//Proceedings of the IEEE International Conference on Multimedia and Expo.Los Alamitos:IEEE Computer Society Press,2009:290-293
[5]Ma X,Zhang J P,Qi C.Hallucinating face by position-patch[J].Pattern Recognition,2010,43(6):2224-2236
[6]Yang J C,Wright J,Huang T,et al.Image super-resolution as sparse representation of raw image patches[C]//Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2008:Article No.4587647
[7]Yang J C,Wright J,Huang T S,et al.Image super-resolution via sparse representation[J].IEEE Transactions on Image Processing,2010,19(11):2861-2873
[8]Jiang J J,Hu R M,Han Z,et al.Position-patch based face hallucination via locality-constrained representation[C]//Proceedings of the 13th IEEE International Conference on Multimedia and Expo.Los Alamitos:IEEE Computer Society Press,2012:212-217
[9]Jiang J J,Hu R M,Wang Z Y,et al.Noise robust face hallucination via locality-constrained representation[J].IEEE Transactions on Multimedia,2014,16(5):1268-1281
[10]Jiang J J,Hu R M,Wang Z Y,et al.Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning[J].IEEE Transactions on Image Processing,2014,23(10):4220-4231
[11]Jiang J J,Hu R M,Wang Z Y,et al.Facial image hallucination through coupled-layer neighbor embedding[J].IEEE Transactions on Circuits and Systems for Video Technology,2016,26(9):1674-1684
[12]Zhang K B,Tao D C,Gao X B,et al.Learning multiple linear mappings for efficient single image super-resolution[J].IEEETransactions on Image Processing,2015,24(3):846-861
[13]Dong C,Loy C C,He K M,et al.Learning a deep convolutional network for image super-resolution[C]//Proceedings of the 13th European Conference on Computer Vision.Aire-la-Ville:Eurographics Association Press,2014:184-199
[14]Dong C,Loy C C,He K M,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307
[15]Kim J,Lee J K,Lee K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the29th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:1646-1654
[16]Sun Yitang,Song Huihui,Zhang Kaihua,et al.Face super-resolution via very deep convolutional neural network[J].Journal of Computer Applications,2018,38(4):1141-1145(in Chinese)(孙毅堂,宋慧慧,张开华,等.基于极深卷积神经网络的人脸超分辨率重建算法[J].计算机应用,2018,38(4):1141-1145)
[17]Thomaz C E,Giraldi G A.A new ranking method for principal components analysis and its application to face image analysis[J].Image and Vision Computing,2010,28(6):902-913
[18]Rowley H A,Baluja S,Kanade T.Neural network-based face detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(1):23-38
[2]Baker S,Kanade T.Limits on super-resolution and how to break them[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2002,24(9):1167-1183
[3]Chang H,Yeung D Y,Xiong Y M.Super-resolution through neighbor embedding[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2004,1:275-282
[4]Ma X,Zhang J P,Qi C.Position-based face hallucination method[C]//Proceedings of the IEEE International Conference on Multimedia and Expo.Los Alamitos:IEEE Computer Society Press,2009:290-293
[5]Ma X,Zhang J P,Qi C.Hallucinating face by position-patch[J].Pattern Recognition,2010,43(6):2224-2236
[6]Yang J C,Wright J,Huang T,et al.Image super-resolution as sparse representation of raw image patches[C]//Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2008:Article No.4587647
[7]Yang J C,Wright J,Huang T S,et al.Image super-resolution via sparse representation[J].IEEE Transactions on Image Processing,2010,19(11):2861-2873
[8]Jiang J J,Hu R M,Han Z,et al.Position-patch based face hallucination via locality-constrained representation[C]//Proceedings of the 13th IEEE International Conference on Multimedia and Expo.Los Alamitos:IEEE Computer Society Press,2012:212-217
[9]Jiang J J,Hu R M,Wang Z Y,et al.Noise robust face hallucination via locality-constrained representation[J].IEEE Transactions on Multimedia,2014,16(5):1268-1281
[10]Jiang J J,Hu R M,Wang Z Y,et al.Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning[J].IEEE Transactions on Image Processing,2014,23(10):4220-4231
[11]Jiang J J,Hu R M,Wang Z Y,et al.Facial image hallucination through coupled-layer neighbor embedding[J].IEEE Transactions on Circuits and Systems for Video Technology,2016,26(9):1674-1684
[12]Zhang K B,Tao D C,Gao X B,et al.Learning multiple linear mappings for efficient single image super-resolution[J].IEEETransactions on Image Processing,2015,24(3):846-861
[13]Dong C,Loy C C,He K M,et al.Learning a deep convolutional network for image super-resolution[C]//Proceedings of the 13th European Conference on Computer Vision.Aire-la-Ville:Eurographics Association Press,2014:184-199
[14]Dong C,Loy C C,He K M,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307
[15]Kim J,Lee J K,Lee K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the29th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:1646-1654
[16]Sun Yitang,Song Huihui,Zhang Kaihua,et al.Face super-resolution via very deep convolutional neural network[J].Journal of Computer Applications,2018,38(4):1141-1145(in Chinese)(孙毅堂,宋慧慧,张开华,等.基于极深卷积神经网络的人脸超分辨率重建算法[J].计算机应用,2018,38(4):1141-1145)
[17]Thomaz C E,Giraldi G A.A new ranking method for principal components analysis and its application to face image analysis[J].Image and Vision Computing,2010,28(6):902-913
[18]Rowley H A,Baluja S,Kanade T.Neural network-based face detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(1):23-38