基于密集神经网络的灰度图像着色算法
|
摘要
针对在灰度图像着色领域中,传统算法信息提取率不高、着色效果不理想的问题,提出了基于密集神经网络的灰度图像着色算法,以实现改善着色效果,让人眼更好地观察图片信息的目的。利用密集神经网络的信息提取高效性,构建并训练了一个端到端的深度学习模型,对图像中的各类信息及特征进行提取。训练网络时与原图像进行对比,以逐渐减小网络输出结果的信息、分类等各类型的损失。训练完成后,只需向网络输入一张灰度图片,即可生成一张颜色饱满、鲜明逼真的彩色图片。实验结果表明,引入密集网络后,可有效改善着色过程中的漏色、细节信息损失、对比度低等问题,所提算法着色效果较基于VGG网络及U-Net、双流网络结构、残差网络(ResNet)等性能优异的先进着色算法而言取得了显著的改进。
Aiming at the problem of low information extraction rate of traditional methods and the unideal coloring effect in the grayscale image colorization field, a grayscale image colorization algorithm based on dense neural network was proposed to improve the colorization effect and make the information of image be better observed by human eyes. With making full use of the high information extraction efficiency of dense neural network, an end-to-end deep learning model was built and trained to extract multiple types of information and features in the image. During the training, the loss of the network output result(such as information loss and classification loss) was gradually reduced by comparing with the original image. After the training, with only a grayscale image input into the trained network, a full and vibrant vivid color image was able to be obtained. The experimental results show that the introduction of dense network can effectively alleviate the problems such as color leakage, loss of detail information and low contrast, during the colorization process. The coloring effect has achieved significant improvement compared with the current advanced coloring methods based on Visual Geometry Group(VGG)-net, U-Net, dual stream network structure, Residual Network(ResNet), etc.
Aiming at the problem of low information extraction rate of traditional methods and the unideal coloring effect in the grayscale image colorization field, a grayscale image colorization algorithm based on dense neural network was proposed to improve the colorization effect and make the information of image be better observed by human eyes. With making full use of the high information extraction efficiency of dense neural network, an end-to-end deep learning model was built and trained to extract multiple types of information and features in the image. During the training, the loss of the network output result(such as information loss and classification loss) was gradually reduced by comparing with the original image. After the training, with only a grayscale image input into the trained network, a full and vibrant vivid color image was able to be obtained. The experimental results show that the introduction of dense network can effectively alleviate the problems such as color leakage, loss of detail information and low contrast, during the colorization process. The coloring effect has achieved significant improvement compared with the current advanced coloring methods based on Visual Geometry Group(VGG)-net, U-Net, dual stream network structure, Residual Network(ResNet), etc.
引文
[1]冈萨雷斯,伍兹.数字图像处理[M].3版.阮秋琦,译.北京:电子工业出版社,2007:484-486.(GONZALEZ R C,WOODS RE.Digital Image Processing[M].3rd ed.RUAN Q Q,translated.Beijing:Publishing House of Electronics Industry,2007:484-486.)
[2]LEVIN A,LISCHINSKI D,WEISS Y.Colorization using optimization[J].ACM Transactions on Graphics,2004,23(3):689-694.
[3]LAGODZINSKI P,SMOLKA B.Medical image colorization[J].Journal of Medical Informatics&Technologies,2007(11):47-57.
[4]LAGODZINSKI P,SMOLKA B.Colorization of medical images[J].China Healthcare Innovation,2009,15(4):13-23.
[5]SHAH A A,MIKITA G,SHAH K M.Medical image colorization using optimization technique[J].Acta Medica Okayama,2013,62(141):235-248.
[6]LI F,ZHU L,ZHANG L,et al.Pseudo-colorization of medical images based on two-stage transfer model[J].Chinese Journal of Stereology and Image Analysis,2013,18(2):135-144.
[7]WELSH T,ASHIKHMIN M,MUELLER K.Transferring color to greyscale images[J].ACM Transactions on Graphics,2002,21(3):277-280.
[8]LIU X,WAN L,QU Y,et al.Intrinsic colorization[J].ACMTransactions on Graphics,2008,27(5):Article No.152.
[9]LIU Y,COHEN M,UYTTENDAELE M,et al.Auto Style:automatic style transfer from image collections to users'images[J].Computer Graphics Forum,2014,33(4):21-31.
[10]MORIMOTO Y,TAGUCHI Y,NAEMURA T.Automatic colorization of grayscale images using multiple images on the Web[C]//Proceedings of ACM SIGGRAPH 2009.New York:ACM,2009:Article No.59.
[11]IRONY R,COHEN-OR D,LISCHINSKI D.Colorization by example[C]//Proceedings of the 16th Eurographics Conference on Rendering Techniques.Aire-la-Ville,Switzerland:Eurographics Association,2005:201-210.
[12]CHENG Z,YANG Q,SHENG B.Deep colorization[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision.Washington,DC:IEEE Computer Society,2015:415-423.
[13]DESHPANDE A,ROCK J,FORSYTH D.Learning large-scale automatic image colorization[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision.Washington,DC:IEEE Computer Society,2015:567-575.
[14]ZHANG R,ISOLA P,EFROS A A.Colorful image colorization[C]//ECCV2016:Proceedings of the 2016 European Conference on Computer Vision.Amsterdam:Springer International Publishing,2016:649-666.
[15]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J/OL].ar Xiv Preprint,2014,2014:ar Xiv.1409.1556(2014-09-04)[2018-08-10].http://arxiv.org/abs/1409.1556.
[16]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolutional networks for biomedical image segmentation[C]//MICCAI2015:Proceedings of the 2015 Medical Image Computing and Computer-Assisted Intervention.Berlin:Springer International Publishing,2015:234-241.
[17]ZHANG R,ZHU J Y,ISOLA P,et al.Real-time user-guided image colorization with learned deep priors[J].ACM Transactions on Graphics,2017,36(4):Article No.119.
[18]LIZUKA S,SIMOSERRA E,ISHIKAWA H.Let there be color!:joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification[J].ACM Transactions on Graphics,2016,35(4):Article No.110.
[19]QIN P L,CHENG Z R,CUI Y H,et al.Research on image colorization algorithm based on residual neural network[C]//CCCV2017:Proceedings of the 2017 CCF Chinese Conference on Computer Vision,CCIS 771.Berlin:Springer,2017:608-621.
[20]HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2016:770-778.
[21]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Image Net classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems.North Miami Beach,FL:Curran Associates Inc.,2012:1097-1105.
[22]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//CVPR 2015:Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2015:1-9.
[23]SRIVASTAVA R K,GREFF K,SCHMIDHUBER J.Highway networks[J/OL].ar Xiv Preprint,2015,2015:ar Xiv.1505.00387(2015-03-03)[2018-08-10].https://arxiv.org/abs/1505.00387.
[24]HUANG G,SUN Y,LIU Z,et al.Deep networks with stochastic depth[C]//ECCV 2016:Proceedings of the 2016 European Conference on Computer Vision.Berlin:Springer,2016:646-661.
[25]LARSSON G,MAIRE M,SHAKHNAROVICH G.Fractal Net:ultra-deep neural networks without residuals[J/OL].ar Xiv Preprint,2016,2016:ar Xiv.1605.07648(2016-05-24)[2018-08-16].https://arxiv.org/abs/1605.07648.
[26]HUANG G,LIU Z,LAURENS V D M,et al.Densely connected convolutional networks[C]//CVPR 2017:Proceedings of the2017 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2017:2261-2269.
[27]ZHOU B,LAPEDRIZA A,XIAO J,et al.Learning deep features for scene recognition using places database[C]//NIPS 2014:Proceedings of the 2014 27th International Conference on Neural Information Processing Systems.Cambridge,CA:MIT Press,2014:487-495.
[28]DENG J,DONG W,SOCHER R,et al.Image Net:a large-scale hierarchical image database[C]//CVPR 2009:Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2009:248-255.
[29]ABADI M,AGARWAL A,BARHAM P,et al.Tensor Flow:large-scale machine learning on heterogeneous distributed systems[J/OL].ar Xiv Preprint,2016,2016:ar Xiv.1603.04467[2018-08-14].https://arxiv.org/abs/1603.04467.
[2]LEVIN A,LISCHINSKI D,WEISS Y.Colorization using optimization[J].ACM Transactions on Graphics,2004,23(3):689-694.
[3]LAGODZINSKI P,SMOLKA B.Medical image colorization[J].Journal of Medical Informatics&Technologies,2007(11):47-57.
[4]LAGODZINSKI P,SMOLKA B.Colorization of medical images[J].China Healthcare Innovation,2009,15(4):13-23.
[5]SHAH A A,MIKITA G,SHAH K M.Medical image colorization using optimization technique[J].Acta Medica Okayama,2013,62(141):235-248.
[6]LI F,ZHU L,ZHANG L,et al.Pseudo-colorization of medical images based on two-stage transfer model[J].Chinese Journal of Stereology and Image Analysis,2013,18(2):135-144.
[7]WELSH T,ASHIKHMIN M,MUELLER K.Transferring color to greyscale images[J].ACM Transactions on Graphics,2002,21(3):277-280.
[8]LIU X,WAN L,QU Y,et al.Intrinsic colorization[J].ACMTransactions on Graphics,2008,27(5):Article No.152.
[9]LIU Y,COHEN M,UYTTENDAELE M,et al.Auto Style:automatic style transfer from image collections to users'images[J].Computer Graphics Forum,2014,33(4):21-31.
[10]MORIMOTO Y,TAGUCHI Y,NAEMURA T.Automatic colorization of grayscale images using multiple images on the Web[C]//Proceedings of ACM SIGGRAPH 2009.New York:ACM,2009:Article No.59.
[11]IRONY R,COHEN-OR D,LISCHINSKI D.Colorization by example[C]//Proceedings of the 16th Eurographics Conference on Rendering Techniques.Aire-la-Ville,Switzerland:Eurographics Association,2005:201-210.
[12]CHENG Z,YANG Q,SHENG B.Deep colorization[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision.Washington,DC:IEEE Computer Society,2015:415-423.
[13]DESHPANDE A,ROCK J,FORSYTH D.Learning large-scale automatic image colorization[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision.Washington,DC:IEEE Computer Society,2015:567-575.
[14]ZHANG R,ISOLA P,EFROS A A.Colorful image colorization[C]//ECCV2016:Proceedings of the 2016 European Conference on Computer Vision.Amsterdam:Springer International Publishing,2016:649-666.
[15]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J/OL].ar Xiv Preprint,2014,2014:ar Xiv.1409.1556(2014-09-04)[2018-08-10].http://arxiv.org/abs/1409.1556.
[16]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolutional networks for biomedical image segmentation[C]//MICCAI2015:Proceedings of the 2015 Medical Image Computing and Computer-Assisted Intervention.Berlin:Springer International Publishing,2015:234-241.
[17]ZHANG R,ZHU J Y,ISOLA P,et al.Real-time user-guided image colorization with learned deep priors[J].ACM Transactions on Graphics,2017,36(4):Article No.119.
[18]LIZUKA S,SIMOSERRA E,ISHIKAWA H.Let there be color!:joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification[J].ACM Transactions on Graphics,2016,35(4):Article No.110.
[19]QIN P L,CHENG Z R,CUI Y H,et al.Research on image colorization algorithm based on residual neural network[C]//CCCV2017:Proceedings of the 2017 CCF Chinese Conference on Computer Vision,CCIS 771.Berlin:Springer,2017:608-621.
[20]HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2016:770-778.
[21]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Image Net classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems.North Miami Beach,FL:Curran Associates Inc.,2012:1097-1105.
[22]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//CVPR 2015:Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2015:1-9.
[23]SRIVASTAVA R K,GREFF K,SCHMIDHUBER J.Highway networks[J/OL].ar Xiv Preprint,2015,2015:ar Xiv.1505.00387(2015-03-03)[2018-08-10].https://arxiv.org/abs/1505.00387.
[24]HUANG G,SUN Y,LIU Z,et al.Deep networks with stochastic depth[C]//ECCV 2016:Proceedings of the 2016 European Conference on Computer Vision.Berlin:Springer,2016:646-661.
[25]LARSSON G,MAIRE M,SHAKHNAROVICH G.Fractal Net:ultra-deep neural networks without residuals[J/OL].ar Xiv Preprint,2016,2016:ar Xiv.1605.07648(2016-05-24)[2018-08-16].https://arxiv.org/abs/1605.07648.
[26]HUANG G,LIU Z,LAURENS V D M,et al.Densely connected convolutional networks[C]//CVPR 2017:Proceedings of the2017 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2017:2261-2269.
[27]ZHOU B,LAPEDRIZA A,XIAO J,et al.Learning deep features for scene recognition using places database[C]//NIPS 2014:Proceedings of the 2014 27th International Conference on Neural Information Processing Systems.Cambridge,CA:MIT Press,2014:487-495.
[28]DENG J,DONG W,SOCHER R,et al.Image Net:a large-scale hierarchical image database[C]//CVPR 2009:Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2009:248-255.
[29]ABADI M,AGARWAL A,BARHAM P,et al.Tensor Flow:large-scale machine learning on heterogeneous distributed systems[J/OL].ar Xiv Preprint,2016,2016:ar Xiv.1603.04467[2018-08-14].https://arxiv.org/abs/1603.04467.