融合去卷积与跳跃嵌套结构的显著性区域检测
|
摘要
针对深度学习的显著性区域检测方法大多存在的显著性图边界信息丢失、轮廓模糊等问题,提出将全局嵌套边缘检测(HED)模型迁移至显著性区域检测任务以增强边界检测,在其基础网络结构之上融入去卷积模块与跳跃嵌套结构,构建了面向显著性区域检测的HED-DSN模型.首先利用去卷积模块以乘积的方式结合底层与高层信息,然后利用跳跃嵌套结构以通道连接的方式将不同层次的特征进行融合,最后用全连接条件随机场对预测得到的显著性图进行优化.在MSRA-B, ECSSD, HKU-IS, SOD和DUT-OMRON共5个数据集上进行实验及模型评价,结果表明,HED-DSN模型在各数据集上均表现良好,不仅能准确地定位出显著性区域,且检测出的区域完整、边界清晰;在客观指标上,该模型的总体性能优于目前最好的DSS模型,且在SOD数据集上提高了近0.7%.
The end to end saliency region detection algorithms based on deep learning always had boundary information loss and contour blurring problems. Addressing these problems, this paper proposes to adopt HED model, which performs well on edge detection tasks to improve edge detection for saliency region. To detect saliency region with distinct areas and sharp edges, this paper proposes to integrate deconvolution module and skip nested architecture on the basis of HED model to construct an advanced model HED-DSN. Firstly, a deconvolution module is introduced to combine the underlying layer information with that of upper layer through multiplying pixel by pixel. Then, a skip nested architecture is employed to combine features from different levels through channel connection way. Finally, the predicted saliency map is optimized with the fully connected conditional random field. Subjective experiments are performed on the 5 most common saliency datasets, including MSRA-B, ECSSD, HKU-IS, SOD, and DUT-OMRON. The results show that the HED-DSN model performs well. Not only it can detect the salient region accurately, but also the detected region is more complete and clear. Objective qualities experiments show that the HED-DSN model is slightly higher than DSS model, one of the best model for saliency region detection, especially is nearly 0.7% higher on the SOD dataset.
The end to end saliency region detection algorithms based on deep learning always had boundary information loss and contour blurring problems. Addressing these problems, this paper proposes to adopt HED model, which performs well on edge detection tasks to improve edge detection for saliency region. To detect saliency region with distinct areas and sharp edges, this paper proposes to integrate deconvolution module and skip nested architecture on the basis of HED model to construct an advanced model HED-DSN. Firstly, a deconvolution module is introduced to combine the underlying layer information with that of upper layer through multiplying pixel by pixel. Then, a skip nested architecture is employed to combine features from different levels through channel connection way. Finally, the predicted saliency map is optimized with the fully connected conditional random field. Subjective experiments are performed on the 5 most common saliency datasets, including MSRA-B, ECSSD, HKU-IS, SOD, and DUT-OMRON. The results show that the HED-DSN model performs well. Not only it can detect the salient region accurately, but also the detected region is more complete and clear. Objective qualities experiments show that the HED-DSN model is slightly higher than DSS model, one of the best model for saliency region detection, especially is nearly 0.7% higher on the SOD dataset.
引文
[1]Hu Y Q,Xie X,Ma W Y,et al.Salient region detection using weighted feature maps based on the human visual attention model[C]//Proceedings of the 5th Pacific Rim Conference on Advances in Multimedia Information Processing.Heidelberg:Springer,2004,3332:993-1000
[2]Itti L.Automatic foveation for video compression using a neurobiological model of visual attention[J].IEEE Transactions on Image Processing,2004,13(10):1304-1318
[3]Guo C L,Zhang L M.A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression[J].IEEE Transactions on Image Processing,2010,19(1):185-198
[4]Cheng M M,Hou Q B,Zhang S H,et al.Intelligent visual media processing:when graphics meets vision[J].Journal of Computer Science and Technology,2017,32(1):110-121
[5]Fu H,Chi Z R,Feng D G.Attention-driven image interpretation with application to image retrieval[J].Pattern Recognition,2006,39(9):1604-1621
[6]Li X H,Lu H C,Zhang L H,et al.Saliency detection via dense and sparse reconstruction[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEEComputer Society Press,2013:2976-2983
[7]Yang C,Zhang L H,Lu H C.Graph-regularized saliency detection with convex-hull-based center prior[J].IEEE Signal Processing Letters,2013,20(7):637-640
[8]Wang L J,Lu H C,Ruan X,et al.Deep networks for saliency detection via local estimation and global search[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:3183-3192
[9]Kim J,Pavlovic V.A shape-based approach for salient object detection using deep learning[C]//Proceedings of the 14th European Conference.Heidelberg:Springer,2016,Part IV:455-470
[10]Li G B,Yu Y Z.Visual saliency based on multiscale deep features[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:5455-5463
[11]Shelhamer E,Long J,Darrell T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(4):640-651
[12]Liu N,Han J W.DHSnet:deep hierarchical saliency network for salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:678-686
[13]Li G B,Yu Y Z.Deep contrast learning for salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:478-487
[14]Xie S N,Tu Z W.Holistically-nested edge detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:1395-1403
[15]Hou Q B,Cheng M M,Hu X W,et al.Deeply supervised salient object detection with short connections[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:5300-5309
[2]Itti L.Automatic foveation for video compression using a neurobiological model of visual attention[J].IEEE Transactions on Image Processing,2004,13(10):1304-1318
[3]Guo C L,Zhang L M.A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression[J].IEEE Transactions on Image Processing,2010,19(1):185-198
[4]Cheng M M,Hou Q B,Zhang S H,et al.Intelligent visual media processing:when graphics meets vision[J].Journal of Computer Science and Technology,2017,32(1):110-121
[5]Fu H,Chi Z R,Feng D G.Attention-driven image interpretation with application to image retrieval[J].Pattern Recognition,2006,39(9):1604-1621
[6]Li X H,Lu H C,Zhang L H,et al.Saliency detection via dense and sparse reconstruction[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEEComputer Society Press,2013:2976-2983
[7]Yang C,Zhang L H,Lu H C.Graph-regularized saliency detection with convex-hull-based center prior[J].IEEE Signal Processing Letters,2013,20(7):637-640
[8]Wang L J,Lu H C,Ruan X,et al.Deep networks for saliency detection via local estimation and global search[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:3183-3192
[9]Kim J,Pavlovic V.A shape-based approach for salient object detection using deep learning[C]//Proceedings of the 14th European Conference.Heidelberg:Springer,2016,Part IV:455-470
[10]Li G B,Yu Y Z.Visual saliency based on multiscale deep features[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2015:5455-5463
[11]Shelhamer E,Long J,Darrell T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(4):640-651
[12]Liu N,Han J W.DHSnet:deep hierarchical saliency network for salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:678-686
[13]Li G B,Yu Y Z.Deep contrast learning for salient object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2016:478-487
[14]Xie S N,Tu Z W.Holistically-nested edge detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2015:1395-1403
[15]Hou Q B,Cheng M M,Hu X W,et al.Deeply supervised salient object detection with short connections[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2017:5300-5309