基于深度学习的图像显著对象检测
|
摘要
显著区域检测可应用在对象识别、图像分割、视频/图像压缩中,是计算机视觉领域的重要研究主题。然而,基于不同视觉显著特征的显著区域检测法常常不能准确地探测出显著对象且计算费时。近来,卷积神经网络模型在图像分析和处理领域取得了极大成功。为提高图像显著区域检测性能,本文提出了一种基于监督式生成对抗网络的图像显著性检测方法。它利用深度卷积神经网络构建监督式生成对抗网络,经生成器网络与鉴别器网络的不断相互对抗训练,使卷积网络准确学习到图像显著区域的特征,进而使生成器输出精确的显著对象分布图。同时,本文将网络自身误差和生成器输出与真值图间的L1距离相结合,来定义监督式生成对抗网络的损失函数,提升了显著区域检测精度。在MSRA10K与ECSSD数据库上的实验结果表明,本文方法分别获得了94.19%与96.24%的准确率和93.99%与90.13%的召回率,F-Measure值也高达94.15%与94.76%,优于先前常用的显著性检测模型。
Salient region detection has been widely applied in object recognition,image segmentation,and image/video compression.It has an important topic in computer vision.However,the saliency detection methods based on different visual salient features often cannot accurately detect salient objects,and they are also time-consuming.Recently,the convolutional neural networks(CNN)-based models have achieved remarkable success in image processing and image analysis.In order to enhance the detection performance of salient objects,this paper presents an image salient object detection approach based on conditional generative adversarial networks(cGAN).This method uses deep CNN to construct the cGAN.By iteratively fighting each other between the generator and the discriminator,the CNN can obtain salient features of images accurately and then urges the generator to produce the high-quality saliency maps.To further improve the saliency detection performance,the loss function of cGAN is defined by cooperating the GAN errors and the L1 distance between the generator outputs and ground truth maps.Experimental results on the MSRA10 K and ECSSD datasets show that the proposed approach outperforms state-of-the-art saliency detection models significantly,and has superior performance with 94.19% and 96.24% in precision,93.99% and 90.13% in recall rate,and 94.15% and 94.76% in F-measure.
Salient region detection has been widely applied in object recognition,image segmentation,and image/video compression.It has an important topic in computer vision.However,the saliency detection methods based on different visual salient features often cannot accurately detect salient objects,and they are also time-consuming.Recently,the convolutional neural networks(CNN)-based models have achieved remarkable success in image processing and image analysis.In order to enhance the detection performance of salient objects,this paper presents an image salient object detection approach based on conditional generative adversarial networks(cGAN).This method uses deep CNN to construct the cGAN.By iteratively fighting each other between the generator and the discriminator,the CNN can obtain salient features of images accurately and then urges the generator to produce the high-quality saliency maps.To further improve the saliency detection performance,the loss function of cGAN is defined by cooperating the GAN errors and the L1 distance between the generator outputs and ground truth maps.Experimental results on the MSRA10 K and ECSSD datasets show that the proposed approach outperforms state-of-the-art saliency detection models significantly,and has superior performance with 94.19% and 96.24% in precision,93.99% and 90.13% in recall rate,and 94.15% and 94.76% in F-measure.
引文
[1] Borji A,Itti L.State-of-the-art in visual attention modeling[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2013,35(1):185-207.
[2] Itti L,Koch C,Niebur E.A model of saliency-based visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,1998,20(11):1254-1259.
[3] Zhang L H,Ai J W,Jiang B W,et al.Saliency detection via absorbing markov chain with learnt transition probability[J].IEEE Transactions on Image Processing,2018,27(2):987-998.
[4] Zhang L H,Yang C,Lu H C,et al.Ranking saliency[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(9):1892-1904.
[5] Bhattacharya S,Venkatesh K S,Gupta S.Visual saliency detection using spatiotemporal decomposition[J].IEEE Transactions on Image Processing,2018,27(4):1665-1675.
[6] Zhai Y,Shah M.Visual attention detection in video sequences using spatiotemporal cues[C].ACM International Conference on Multimedia.ACM,2006,478-482.
[7] Zhang L Q,Hou X D.Saliency detection:A spectral residual approach[C].IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2007).IEEE,2007,1-8.
[8] Achanta R,Hemami S,Estrada F,Susstrunk S.Frequency-tuned salient region detection[C].IEEE International conference on computer vision and pattern recognition (CVPR 2009).2009,1597-1604.
[9] Jiang H Z,Wang J D,Yuan Z J,et al.Salient object detection:A discriminative regional feature integration approach[C].IEEE Conference on Computer Vision & Pattern Recognition (CVPR 2013).2013,2083-2090.
[10] Yan Q,Xu L,Shi J P,et al.Hierarchical saliency detection[C].IEEE Conference on Computer Vision & Pattern Recognition (CVPR 2013).2013,1155-1162.
[11] Zhu W J,Liang S,Wei Y C,et al.Saliency optimization from robust background detection[C].IEEE Conference on Computer Vision & Pattern Recognition (CVPR 2014).2014,2814-2821.
[12] Wu X,Wang H,Chen W.Saliency detection based on graph and independent component analysis with reference[C].International Conference on Control Automation Robotics & Vision.IEEE,2015,1207-1212.
[13] Scharfenberger C,Jain A,Wong A,et al.Image saliency detection via multi-scale statistical non-redundancy modeling[C].IEEE International Conference on Image Processing,IEEE,2015,4294-4298.
[14] Zhou L,Yang Z,Yuan Q,et al.Salient region detection via integrating diffusion-based compactness and local contrast[J].IEEE Transactions on Image Processing,2015,24(11):3308-3320.
[15] Pan H,Jiang H.A deep learning based fast image saliency detection algorithm[J].arXiv:1602.00577,2016.
[16] Yang Y,Sheng B,Wu W,et al.Image saliency detection based on rectangular-wave spectrum analysis[J].Multimedia Tools & Applications,2016,75(11):6173-6187.
[17] Li G B,Yu Y Z.Visual saliency detection based on multiscale deep CNN features[J].IEEE Transactions on Image Processing,2016,25(11):5012-5024.
[18] Lee G,Tai Y W,Kim J.Deep saliency with encoded low level distance map and high level features[C].IEEE Conference on Computer Vision & Pattern Recognition (CVPR 2016).IEEE,2016,660-668.
[19] Liu N,Han J W.DHSNet:Deep hierarchical saliency network for salient object detection[C].IEEE Conference on Computer Vision & Pattern Recognition.IEEE,2016,678-686.
[20] Liu N,Han J W.A deep spatial contextual long-term recurrent convolutional network for saliency detection[J].IEEE Transactions on Image Processing,2018,27(7):3264-3274.
[21] LIU Feng,SHEN Tong-sheng,LOU Shu-li,et al.Deep network saliency detection based on global model and local optimization[J].Acta Optica Sinica,2017,37(12):264-272.刘峰,沈同圣,娄树理,等.全局模型和局部优化的深度网络显著性检测[J].光学学报,2017,37(12):264-272.
[22] U-Net:Convolutional networks for biomedical image segmentation.https://lmb.informatik.uni-freiburg.de/ people/ronneber/ u-net/
[23] Cheng M M,Zhang G X,Mitra N J,et al.Global contrast based salient region detection[C].IEEE Conference on Computer Vision & Pattern Recognition (CVPR 2011),IEEE,2011,409-416.
[24] THUR15000 dataset. [Online]. http: //mmcheng. net/gsal/
[25] Saliency object detection:a benchmark.[Online].http://mmcheng.net/zh/salobjbenchmark/
[2] Itti L,Koch C,Niebur E.A model of saliency-based visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,1998,20(11):1254-1259.
[3] Zhang L H,Ai J W,Jiang B W,et al.Saliency detection via absorbing markov chain with learnt transition probability[J].IEEE Transactions on Image Processing,2018,27(2):987-998.
[4] Zhang L H,Yang C,Lu H C,et al.Ranking saliency[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(9):1892-1904.
[5] Bhattacharya S,Venkatesh K S,Gupta S.Visual saliency detection using spatiotemporal decomposition[J].IEEE Transactions on Image Processing,2018,27(4):1665-1675.
[6] Zhai Y,Shah M.Visual attention detection in video sequences using spatiotemporal cues[C].ACM International Conference on Multimedia.ACM,2006,478-482.
[7] Zhang L Q,Hou X D.Saliency detection:A spectral residual approach[C].IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2007).IEEE,2007,1-8.
[8] Achanta R,Hemami S,Estrada F,Susstrunk S.Frequency-tuned salient region detection[C].IEEE International conference on computer vision and pattern recognition (CVPR 2009).2009,1597-1604.
[9] Jiang H Z,Wang J D,Yuan Z J,et al.Salient object detection:A discriminative regional feature integration approach[C].IEEE Conference on Computer Vision & Pattern Recognition (CVPR 2013).2013,2083-2090.
[10] Yan Q,Xu L,Shi J P,et al.Hierarchical saliency detection[C].IEEE Conference on Computer Vision & Pattern Recognition (CVPR 2013).2013,1155-1162.
[11] Zhu W J,Liang S,Wei Y C,et al.Saliency optimization from robust background detection[C].IEEE Conference on Computer Vision & Pattern Recognition (CVPR 2014).2014,2814-2821.
[12] Wu X,Wang H,Chen W.Saliency detection based on graph and independent component analysis with reference[C].International Conference on Control Automation Robotics & Vision.IEEE,2015,1207-1212.
[13] Scharfenberger C,Jain A,Wong A,et al.Image saliency detection via multi-scale statistical non-redundancy modeling[C].IEEE International Conference on Image Processing,IEEE,2015,4294-4298.
[14] Zhou L,Yang Z,Yuan Q,et al.Salient region detection via integrating diffusion-based compactness and local contrast[J].IEEE Transactions on Image Processing,2015,24(11):3308-3320.
[15] Pan H,Jiang H.A deep learning based fast image saliency detection algorithm[J].arXiv:1602.00577,2016.
[16] Yang Y,Sheng B,Wu W,et al.Image saliency detection based on rectangular-wave spectrum analysis[J].Multimedia Tools & Applications,2016,75(11):6173-6187.
[17] Li G B,Yu Y Z.Visual saliency detection based on multiscale deep CNN features[J].IEEE Transactions on Image Processing,2016,25(11):5012-5024.
[18] Lee G,Tai Y W,Kim J.Deep saliency with encoded low level distance map and high level features[C].IEEE Conference on Computer Vision & Pattern Recognition (CVPR 2016).IEEE,2016,660-668.
[19] Liu N,Han J W.DHSNet:Deep hierarchical saliency network for salient object detection[C].IEEE Conference on Computer Vision & Pattern Recognition.IEEE,2016,678-686.
[20] Liu N,Han J W.A deep spatial contextual long-term recurrent convolutional network for saliency detection[J].IEEE Transactions on Image Processing,2018,27(7):3264-3274.
[21] LIU Feng,SHEN Tong-sheng,LOU Shu-li,et al.Deep network saliency detection based on global model and local optimization[J].Acta Optica Sinica,2017,37(12):264-272.刘峰,沈同圣,娄树理,等.全局模型和局部优化的深度网络显著性检测[J].光学学报,2017,37(12):264-272.
[22] U-Net:Convolutional networks for biomedical image segmentation.https://lmb.informatik.uni-freiburg.de/ people/ronneber/ u-net/
[23] Cheng M M,Zhang G X,Mitra N J,et al.Global contrast based salient region detection[C].IEEE Conference on Computer Vision & Pattern Recognition (CVPR 2011),IEEE,2011,409-416.
[24] THUR15000 dataset. [Online]. http: //mmcheng. net/gsal/
[25] Saliency object detection:a benchmark.[Online].http://mmcheng.net/zh/salobjbenchmark/