自适应编辑传播的人脸图像光照迁移
|
摘要
提出一种融合人脸特征分析的自适应编辑传播方法来提高人脸图像光照渲染的效能,并以此实现复杂背景下基于单张参考人脸的自动光照迁移。该方法把参考图像的人脸区域与目标图像的背景区域进行融合,利用边缘保持平滑滤波从融合的人脸中提取光照信息。然后,构建一个能随不同人脸区域而自适应变化的编辑传播模型,把提取的光照信息从人脸区域扩散到背景区域,生成光照模板。最后,通过融合光照模板与目标人脸实现光照迁移。在YaleB数据库的定量实验中,平均每张迁移光照人脸有超过85%的像素(归一化到[0,255])与标准光照人脸的像素值差异小于6。与其他方法对比,本文方法获得的光照渲染效果具有更好一致性。结果表明,本文方法扩展了光照迁移的适用范围,具有良好的稳定性,能在具有不同性别和背景等特点的参考人脸与目标人脸中生成自然的光照迁移效果。
An adaptive edit propagation method based on facial priors was proposed to achieve natural relighting effect of a portrait in a complex background using a single reference face.Firstly,the facial region of a reference image and the background region of a target were combined,and an edgepreserving smoothing filter was used to extract the illumination information from the combined image.Then a new edit propagation model adaptively changed with facial parameters was constructed to generate an illumination template by propagating the illumination from the facial region to the background.Finally,the illumination template and the target were multiplied in the luminance channel to achieve the relighting effect.The quantitative experiments in YaleB database show that there are averagely over 85% pixels(normalized to [0,255])in a relighting effect face,whose intensity differences are less than 6 comparing with the ground true.As compared with other methods,the relighting effects of proposed method are more consistency.The conclusion shows that the proposed method achieves reliable and natural face relighting effect on portraits with different genders and backgrounds.
An adaptive edit propagation method based on facial priors was proposed to achieve natural relighting effect of a portrait in a complex background using a single reference face.Firstly,the facial region of a reference image and the background region of a target were combined,and an edgepreserving smoothing filter was used to extract the illumination information from the combined image.Then a new edit propagation model adaptively changed with facial parameters was constructed to generate an illumination template by propagating the illumination from the facial region to the background.Finally,the illumination template and the target were multiplied in the luminance channel to achieve the relighting effect.The quantitative experiments in YaleB database show that there are averagely over 85% pixels(normalized to [0,255])in a relighting effect face,whose intensity differences are less than 6 comparing with the ground true.As compared with other methods,the relighting effects of proposed method are more consistency.The conclusion shows that the proposed method achieves reliable and natural face relighting effect on portraits with different genders and backgrounds.
引文
[1]BLANZ V,VETTER T.A morphable model for the synthesis of 3Dfaces[C].Proc.Conf.CGIT,1999:187-194.
[2]BASRI R,JACOBS D W.Lambertian reflectance and linear subspaces[J].IEEE Trans.Patt.Anal.Mach.Intell.,2003,25(2):218-233.
[3]SHASHUA A,RIKLIN-RAVIV T.The quotient image:class-based re-rendering and recognition with varying illuminations[J].IEEE Trans.Patt.Anal.Mach.Intell.,2001,23(2):129-139.
[4]PEERS P,TAMURA N,MATUSIK W,et al..Post-production facial performance relighting using reflectance transfer[J].ACM Trans.Graph.,2007,26(3):10.
[5]LI Q,YIN W,DENG Z.Image-based face illumination transferring using logarithmic total variation models[J].The Visual Computer,2010,26(1):41-49.
[6]CHEN J,SU G,HE J,et al..Face image relighting using locally constrained global optimization[C].Proc.ECCV,2010:44-57.
[7]CHEN X,WU H,JIN X,et al..Face illumination manipulation using a single reference image by adaptive layer decomposition[J].IEEE Transactions on Image Processing,2013,22(11):4249-4259.
[8]LIANG L,JIN L.A new face relighting method based on edge-preserving filter[J].IEICE Transactions on Information and Systems,2013,E96-D(12):2904-2907.
[9]CHEN X,JIN X,ZHAO Q,et al..Artistic illumination transfer for portraits[J].Comput.Graphics Forum,2012,31(4):1425-1434.
[10]CHEN X,ZOU D,ZHAO Q,et al..Manifold preserving edit propagation[J].ACM Trans.Graph.,2012,31(6):132.
[11]LEVIN A,LISCHINSKI D,WEISS Y.Colorization using optimization[J].ACM Trans.Graph.,2004,23(3):689-694.
[12]厉旭杰,赵汉理,黄辉.局部线性模型优化的灰度图像彩色化[J].中国图像图形学报,2013,18(4):460-466.LI X J,ZHAO H L,HUANG H.Local linear model optimization based grayscale image colorization[J].Journal of Image and Graphics,2013,18(4):460-466.(in Chinese)
[13]LIANG L,JIN L,LI X.Facial skin beautification using adaptive region-aware masks[J].IEEE Trans.on Cybernetics,2014,44(12):2600-2612.
[14]CHEN X,ZOU D,LI J,et al..Sparse dictionary learning for edit propagation of high-resolution images[C].Proc.CVPR,2014:2854-2861.
[15]LISCHINSKI D,FARBMAN Z,UYTTENDAELE M,et al..Interactive local adjustment of tonal values[J].ACM Trans.Graph.,2006,25(3):646-653.
[16]杨利平,辜小花.用于人脸识别的相对梯度直方图特征描述[J].光学精密工程,2014,22(1):152-159.YANG L P,GU X H.Relative gradient histogram features for face recognition[J].Opt.Precision Eng.,2014,22(1):152-159.(in Chinese)
[17]COOTES T F,TAYLOR C J,COOPER D H,et al..Active shape models-their training and application[J].Comput.Vision Image Understand,1995,61(1):38-59.
[18]LEE S,WOLBERG G,SHIN S Y.Scattered data interpolation with multilevel B-splines[J].IEEE Trans.Vis.Comput.Graphics,1997,3(3):228-244.
[19]FARBMAN Z,FATTAL R,LISCHINSKI D,et al..Edge-preserving decompositions for multiscale tone and detail manipulation[J].ACM Trans.Graph.,2008,27(3):10.
[20]GEORGHIADES A,BELHUMEUR P,KRIEGMAN D.From few to many:Illumination cone models for face recognition under variable lighting and pose[J].IEEE Trans.Pattern Anal.Mach.Intell.,2001,23(6):643-660.
[21]Caltech Face Database[EB/OL].Available:http://www.vision.caltech.edu/html-files/.
[22]邓承志,田伟,汪胜前,等.近似稀疏正则化的红外图像超分辨率重建[J].光学精密工程,2014,22(6):1648-1654.DENG CH ZH,TIAN W,WANG SH Q,et al..Super-resolution reconstruction of approximate sparsity regularized infrared images[J].Opt.Precision Eng.,2014,22(6):1648-1654.(in Chinese)
[23]BYCHKOVSKY V,PARIS S,CHAN E,et al..Learning photographic global tonal adjustment with a database of input/output image pairs[C].Proc.CVPR,2011:97-104.
[24]WANG X,TANG X.Face photo-sketch synthesis and recognition[J].IEEE Trans.Patt.Anal.Mach.Intell.,2009,31(11):1955-1967.
[25]李伟红,朱宪宇,龚卫国.基于人脸画像的伪照片合成及修正[J].光学精密工程,2014,22(5):1371-1378.LI W H,ZHU X Y,GONG W G.Pseudophoto synthesis based on face sketch and its amendment[J].Opt.Precision Eng.,2014,22(5):1371-1378.(in Chinese)
[2]BASRI R,JACOBS D W.Lambertian reflectance and linear subspaces[J].IEEE Trans.Patt.Anal.Mach.Intell.,2003,25(2):218-233.
[3]SHASHUA A,RIKLIN-RAVIV T.The quotient image:class-based re-rendering and recognition with varying illuminations[J].IEEE Trans.Patt.Anal.Mach.Intell.,2001,23(2):129-139.
[4]PEERS P,TAMURA N,MATUSIK W,et al..Post-production facial performance relighting using reflectance transfer[J].ACM Trans.Graph.,2007,26(3):10.
[5]LI Q,YIN W,DENG Z.Image-based face illumination transferring using logarithmic total variation models[J].The Visual Computer,2010,26(1):41-49.
[6]CHEN J,SU G,HE J,et al..Face image relighting using locally constrained global optimization[C].Proc.ECCV,2010:44-57.
[7]CHEN X,WU H,JIN X,et al..Face illumination manipulation using a single reference image by adaptive layer decomposition[J].IEEE Transactions on Image Processing,2013,22(11):4249-4259.
[8]LIANG L,JIN L.A new face relighting method based on edge-preserving filter[J].IEICE Transactions on Information and Systems,2013,E96-D(12):2904-2907.
[9]CHEN X,JIN X,ZHAO Q,et al..Artistic illumination transfer for portraits[J].Comput.Graphics Forum,2012,31(4):1425-1434.
[10]CHEN X,ZOU D,ZHAO Q,et al..Manifold preserving edit propagation[J].ACM Trans.Graph.,2012,31(6):132.
[11]LEVIN A,LISCHINSKI D,WEISS Y.Colorization using optimization[J].ACM Trans.Graph.,2004,23(3):689-694.
[12]厉旭杰,赵汉理,黄辉.局部线性模型优化的灰度图像彩色化[J].中国图像图形学报,2013,18(4):460-466.LI X J,ZHAO H L,HUANG H.Local linear model optimization based grayscale image colorization[J].Journal of Image and Graphics,2013,18(4):460-466.(in Chinese)
[13]LIANG L,JIN L,LI X.Facial skin beautification using adaptive region-aware masks[J].IEEE Trans.on Cybernetics,2014,44(12):2600-2612.
[14]CHEN X,ZOU D,LI J,et al..Sparse dictionary learning for edit propagation of high-resolution images[C].Proc.CVPR,2014:2854-2861.
[15]LISCHINSKI D,FARBMAN Z,UYTTENDAELE M,et al..Interactive local adjustment of tonal values[J].ACM Trans.Graph.,2006,25(3):646-653.
[16]杨利平,辜小花.用于人脸识别的相对梯度直方图特征描述[J].光学精密工程,2014,22(1):152-159.YANG L P,GU X H.Relative gradient histogram features for face recognition[J].Opt.Precision Eng.,2014,22(1):152-159.(in Chinese)
[17]COOTES T F,TAYLOR C J,COOPER D H,et al..Active shape models-their training and application[J].Comput.Vision Image Understand,1995,61(1):38-59.
[18]LEE S,WOLBERG G,SHIN S Y.Scattered data interpolation with multilevel B-splines[J].IEEE Trans.Vis.Comput.Graphics,1997,3(3):228-244.
[19]FARBMAN Z,FATTAL R,LISCHINSKI D,et al..Edge-preserving decompositions for multiscale tone and detail manipulation[J].ACM Trans.Graph.,2008,27(3):10.
[20]GEORGHIADES A,BELHUMEUR P,KRIEGMAN D.From few to many:Illumination cone models for face recognition under variable lighting and pose[J].IEEE Trans.Pattern Anal.Mach.Intell.,2001,23(6):643-660.
[21]Caltech Face Database[EB/OL].Available:http://www.vision.caltech.edu/html-files/.
[22]邓承志,田伟,汪胜前,等.近似稀疏正则化的红外图像超分辨率重建[J].光学精密工程,2014,22(6):1648-1654.DENG CH ZH,TIAN W,WANG SH Q,et al..Super-resolution reconstruction of approximate sparsity regularized infrared images[J].Opt.Precision Eng.,2014,22(6):1648-1654.(in Chinese)
[23]BYCHKOVSKY V,PARIS S,CHAN E,et al..Learning photographic global tonal adjustment with a database of input/output image pairs[C].Proc.CVPR,2011:97-104.
[24]WANG X,TANG X.Face photo-sketch synthesis and recognition[J].IEEE Trans.Patt.Anal.Mach.Intell.,2009,31(11):1955-1967.
[25]李伟红,朱宪宇,龚卫国.基于人脸画像的伪照片合成及修正[J].光学精密工程,2014,22(5):1371-1378.LI W H,ZHU X Y,GONG W G.Pseudophoto synthesis based on face sketch and its amendment[J].Opt.Precision Eng.,2014,22(5):1371-1378.(in Chinese)