结合边缘模糊的景深显著性在图像缩放中的研究
|
摘要
针对内容敏感图像缩放技术调整图像尺寸时会出现重要信息丢失、物体边缘扭曲、非显著区域结构信息不完整等问题,提出基于边缘模糊的景深显著性算法.根据心理学原理,引入非显著区域结构信息保持的概念.采用边缘模糊特性进行景深估算,通过学习得到模糊字典提出模糊景深描述子,并以模糊景深描述子辅助进行显著性计算;为了达到在图像缩放时保持结构均衡,通过改进传统的区域型显著计算,提出显著性边缘分布特征,能够在保持显著信息、边缘完整的同时,有效减少结构信息的丢失或损坏,使缩放后的图像更符合人们的视觉体验.通过与经典算法和最新算法的对比实验,验证了文中算法的有效性.
Image retargeting technique becomes research hot with the widely used digital images on different display screens, such as TV, i Phone, i Pad and other wearable equipment. The objectives of image retargeting are important information preservation, less edge distortion during increasing/decreasing image size. The major existed content-aware methods perform well. However, there are two problems should be improved: the slight distortion appeared at the object edges and the structure distortion in the non-salient area. In order to solve the problems above, we propose a novel algorithm to calculate depth-based saliency based on the edge blur. According to psychological theories, people evaluate image quality based on multi-level judgments and comparison between different areas, both image content and structure. The paper proposes a new standard: the structure preserving in non-salient area. After observation and image analysis, blur(slight blur) is generally existed at the edge of objects. The blur feature is used to estimate the depth cue, named blur depth descriptor. It can be used in the process of saliency computation for balanced image retargeting results. In order to keep the structure information in non-salient area, the salient edge map is presented in Seam Carving process, instead of field-based saliency computation. The derivative blur-based depth from x-and y-direction can avoid the redundant energy seam around salient objects causing structure distortion. There are three comparison experiments between the proposed algorithm and selected classic algorithms: salient edge preservation experiment, large scale retargeting on high-resolution images experiment and average energy comparison. The promising results prove the feasibility of the proposed algorithm.
Image retargeting technique becomes research hot with the widely used digital images on different display screens, such as TV, i Phone, i Pad and other wearable equipment. The objectives of image retargeting are important information preservation, less edge distortion during increasing/decreasing image size. The major existed content-aware methods perform well. However, there are two problems should be improved: the slight distortion appeared at the object edges and the structure distortion in the non-salient area. In order to solve the problems above, we propose a novel algorithm to calculate depth-based saliency based on the edge blur. According to psychological theories, people evaluate image quality based on multi-level judgments and comparison between different areas, both image content and structure. The paper proposes a new standard: the structure preserving in non-salient area. After observation and image analysis, blur(slight blur) is generally existed at the edge of objects. The blur feature is used to estimate the depth cue, named blur depth descriptor. It can be used in the process of saliency computation for balanced image retargeting results. In order to keep the structure information in non-salient area, the salient edge map is presented in Seam Carving process, instead of field-based saliency computation. The derivative blur-based depth from x-and y-direction can avoid the redundant energy seam around salient objects causing structure distortion. There are three comparison experiments between the proposed algorithm and selected classic algorithms: salient edge preservation experiment, large scale retargeting on high-resolution images experiment and average energy comparison. The promising results prove the feasibility of the proposed algorithm.
引文
[1]Avidan S,Shamir A.Seam carving for content-aware image resizing[J].ACM Transactions on Graphics,2007,26(3):Article No.10
[2]Rubinstein M,Shamir A,Avidan S.Improved seam carving for video retargeting[J].ACM Transactions on Graphics,2008,27(3):Article No.16
[3]Achanta R,Süsstrunk S.Saliency detection for content-aware image resizing[C]//Proceedings of the 16th IEEE International Conference on Image Processing.Los Alamitos:IEEE Computer Society Press,2009:1005-1008
[4]Shen J B,Wang D P,Li X L.Depth-aware image seam carving[J].IEEE Transactions on Cybernetics,2013,43(5):1453-1461
[5]Zhou B,Wang X Y,Cao S X,et al.Optimal bi-directional seam carving for compressibility-aware image retargeting[J].Journal of Visual Communication and Image Representation,2016,41:21-30
[6]Shafieyan F,Karimi N,Mirmahboub B,et al.Image retargeting using depth assisted saliency map[J].Signal Processing Image Communication,2017,50:34-43
[7]Nie Dongdong,Ma Lizhuang.Improved image seam carving method[J].Journal of Computer-Aided Design&Computer Graphics,2011,23(11):1890-1895(in Chinese)(聂栋栋,马利庄.改进的图像缝雕刻算法[J].计算机辅助设计与图形学学报,2011,23(11):1890-1895)
[8]Chen J H,Miao L F,Liu X G.Image re-targeting with balanced energy map and foreground constraint[J].World Wide Web,2011,14(3):281-292
[9]Toony Z,Jamzad M.A modified saliency detection for content-aware image resizing using cellular automata[C]//Proceedings of International Conference on Signal and Image Processing.Los Alamitos:IEEE Computer Society Press,2010:175-179
[10]Cui Puming,Liu Xiuping,Wang Xiaochao,et al.A contentaware image retargeting method based on mesh deformation[J].Journal of Computer-Aided Design&Computer Graphics,2013,25(6):788-793(in Chinese)(崔普明,刘秀平,王小超,等.一种基于网格变形的图像放缩方法[J].计算机辅助设计与图形学学报,2013,25(6):788-793)
[11]He Z W,Gao M Y,Yu H B,et al.A new improved seam carving content aware image resizing method[C]//Proceedings of the 8th IEEE Conference on Industrial Electronics and Applications.Los Alamitos:IEEE Computer Society Press,2013:738-741
[12]Yang Bo,Liu Jian,Chen Jie,et al.Rectangling panoramic images via sub-pixel seam carving[J].Journal of Computer-Aided Design&Computer Graphics,2016,28(10):1723-1731(in Chinese)(杨博,刘建,陈杰,等.采用亚像素缝雕刻的图像整形算法[J].计算机辅助设计与图形学学报,2016,28(10):1723-1731)
[13]Shi M L,Peng G Q,Yang L,et al.Optimal bi-directional seam carving for content-aware image resizing[M]//Lecture Notes in Computer Science.Heidelberg:Springer,2010,6249:456-467
[14]Rubinstein M,Shamir A,Avidan S.Multi-operator media retargeting[J].ACM Transactions on Graphics,2009,28(3):Article No.23
[15]Dong W M,Zhou N,Paul J C,et al.Optimized image resizing using seam carving and scaling[J].ACM Transactions on Graphics,2009,28(5):Article No.125
[16]Shafieyan F,Karimi N,Mirmahboub B,et al.Image seam carving using depth assisted saliency map[C]//Proceedings of IEEE International Conference on Image Processing.Los Alamitos:IEEE Computer Society Press,2014:1155-1159
[17]Sch?lkopf B,Platt J,Hofmann T.Graph-based visual saliency[C]//Proceedings of Advances in Neural Information Processing Systems.Cambridge:MIT Press,2006:545-552
[18]Itti L,Koch C,Niebur E.A model of saliency-based visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259
[19]Yan Q,Xu L,Shi J P,et al.Hierarchical saliency detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2013:1155-1162
[20]Shi J P,Xu L,Jia J Y.Just noticeable defocus blur detection and estimation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:657-665
[21]Wang Y S,Tai C L,Sorkine O,et al.Optimized scale-andstretch for image resizing[J].ACM Transactions on Graphics,2008,27(5):Article No.118
[2]Rubinstein M,Shamir A,Avidan S.Improved seam carving for video retargeting[J].ACM Transactions on Graphics,2008,27(3):Article No.16
[3]Achanta R,Süsstrunk S.Saliency detection for content-aware image resizing[C]//Proceedings of the 16th IEEE International Conference on Image Processing.Los Alamitos:IEEE Computer Society Press,2009:1005-1008
[4]Shen J B,Wang D P,Li X L.Depth-aware image seam carving[J].IEEE Transactions on Cybernetics,2013,43(5):1453-1461
[5]Zhou B,Wang X Y,Cao S X,et al.Optimal bi-directional seam carving for compressibility-aware image retargeting[J].Journal of Visual Communication and Image Representation,2016,41:21-30
[6]Shafieyan F,Karimi N,Mirmahboub B,et al.Image retargeting using depth assisted saliency map[J].Signal Processing Image Communication,2017,50:34-43
[7]Nie Dongdong,Ma Lizhuang.Improved image seam carving method[J].Journal of Computer-Aided Design&Computer Graphics,2011,23(11):1890-1895(in Chinese)(聂栋栋,马利庄.改进的图像缝雕刻算法[J].计算机辅助设计与图形学学报,2011,23(11):1890-1895)
[8]Chen J H,Miao L F,Liu X G.Image re-targeting with balanced energy map and foreground constraint[J].World Wide Web,2011,14(3):281-292
[9]Toony Z,Jamzad M.A modified saliency detection for content-aware image resizing using cellular automata[C]//Proceedings of International Conference on Signal and Image Processing.Los Alamitos:IEEE Computer Society Press,2010:175-179
[10]Cui Puming,Liu Xiuping,Wang Xiaochao,et al.A contentaware image retargeting method based on mesh deformation[J].Journal of Computer-Aided Design&Computer Graphics,2013,25(6):788-793(in Chinese)(崔普明,刘秀平,王小超,等.一种基于网格变形的图像放缩方法[J].计算机辅助设计与图形学学报,2013,25(6):788-793)
[11]He Z W,Gao M Y,Yu H B,et al.A new improved seam carving content aware image resizing method[C]//Proceedings of the 8th IEEE Conference on Industrial Electronics and Applications.Los Alamitos:IEEE Computer Society Press,2013:738-741
[12]Yang Bo,Liu Jian,Chen Jie,et al.Rectangling panoramic images via sub-pixel seam carving[J].Journal of Computer-Aided Design&Computer Graphics,2016,28(10):1723-1731(in Chinese)(杨博,刘建,陈杰,等.采用亚像素缝雕刻的图像整形算法[J].计算机辅助设计与图形学学报,2016,28(10):1723-1731)
[13]Shi M L,Peng G Q,Yang L,et al.Optimal bi-directional seam carving for content-aware image resizing[M]//Lecture Notes in Computer Science.Heidelberg:Springer,2010,6249:456-467
[14]Rubinstein M,Shamir A,Avidan S.Multi-operator media retargeting[J].ACM Transactions on Graphics,2009,28(3):Article No.23
[15]Dong W M,Zhou N,Paul J C,et al.Optimized image resizing using seam carving and scaling[J].ACM Transactions on Graphics,2009,28(5):Article No.125
[16]Shafieyan F,Karimi N,Mirmahboub B,et al.Image seam carving using depth assisted saliency map[C]//Proceedings of IEEE International Conference on Image Processing.Los Alamitos:IEEE Computer Society Press,2014:1155-1159
[17]Sch?lkopf B,Platt J,Hofmann T.Graph-based visual saliency[C]//Proceedings of Advances in Neural Information Processing Systems.Cambridge:MIT Press,2006:545-552
[18]Itti L,Koch C,Niebur E.A model of saliency-based visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259
[19]Yan Q,Xu L,Shi J P,et al.Hierarchical saliency detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2013:1155-1162
[20]Shi J P,Xu L,Jia J Y.Just noticeable defocus blur detection and estimation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:657-665
[21]Wang Y S,Tai C L,Sorkine O,et al.Optimized scale-andstretch for image resizing[J].ACM Transactions on Graphics,2008,27(5):Article No.118