基于循环生成对抗网络的道路场景语义分割
|
摘要
在无人驾驶技术中,道路场景语义分割是一个非常重要的环境感知任务。传统的基于深度学习方法需要大量像素级标注样本,限制了应用范围。本文提出一种基于循环生成对抗网络的道路场景语义分割方法,无需成对数据也可实现图像语义分割,降低对数据集的要求;使用L2范数和最小二乘损失方法解决训练过程中出现的模式崩溃现象,增加了训练过程的稳定性,并提高了图像分割的质量。为了验证本文方法的有效性,在常用的道路场景数据集进行实验,结果显示该方法的分割精确度有明显提高。
In driverless technology, semantic segmentation of road scenes is a very important environmental perception task. Traditional deep learning methods require a large number of pixel-level annotation samples, which limits the scope of application. In this paper, we proposed a semantic segmentation method for road scenes based on cycle-consistent adversarial network. Image semantic segmentation was realized and the dataset requirements were reduced without paired data. Meanwhile,the collapse of mode during training was solved, and the stability of the training process and the quality of generated images was improved by using L2 norm and least square loss method. In order to validate the effectiveness of the proposed method, experiments were performed on commonly used road scene datasets. The results show that the accuracy of the proposed method is obviously improved.
In driverless technology, semantic segmentation of road scenes is a very important environmental perception task. Traditional deep learning methods require a large number of pixel-level annotation samples, which limits the scope of application. In this paper, we proposed a semantic segmentation method for road scenes based on cycle-consistent adversarial network. Image semantic segmentation was realized and the dataset requirements were reduced without paired data. Meanwhile,the collapse of mode during training was solved, and the stability of the training process and the quality of generated images was improved by using L2 norm and least square loss method. In order to validate the effectiveness of the proposed method, experiments were performed on commonly used road scene datasets. The results show that the accuracy of the proposed method is obviously improved.
引文
[1]刘富强,张姗姗,朱文红,等.一种基于视觉的车道线检测与跟踪算法[J].同济大学学报(自然科学版),2010,38(2):223-229.DOI:10.3969/j.issn.0253-374x.2010.02.013.LIU F Q,ZHANG S S,ZHU W H,et al.A visionbased lane detection and tracking algorithm[J].Journal of Tongji University(Natural Science),2010,38(2):223-229.DOI:10.3969/j.issn.0253-374x.2010.02.013(Ch).
[2]施培蓓,刘贵全,汪中.基于快速增量学习的行人检测方法[J].小型微型计算机系统,2015,36(8):1837-1841.SHI P B,LIU G Q,WANG Z.Fast incremental learning method for pedestrian detection[J].Journal of Chinese Computer Systems,2015,36(8):1837-1841(Ch).
[3]徐岩,王权威,韦镇余.一种融合加权ELM和AdaBoost的交通标志识别算法[J].小型微型计算机系统,2017,38(9):2028-2032.DOI:10.3969/j.issn.1000-1220.2017.09.021.XU Y,WANG Q W,WEI Z Y.Traffic sign recognition algorithm combining weighted ELM and AdaBoost[J].Journal of Chinese Computer Systems,2017,38(9):2028-2032.DOI:10.3969/j.issn.1000-1220.2017.09.021.(Ch).
[4]魏云超,赵耀.基于DCNN的图像语义分割综述[J].北京交通大学学报,2016,40(4):82-91.DOI:10.11860/j.issn.1673-0291.2016.04.013.WEI Y C ZHAO Y.A review on image semantic segmentation based on DCNN[J].Journal of Beijing Jiaotong University,2016,40(4):82-91.DOI:10.11860/j.issn.1673-0291.2016.04.013(Ch).
[5]姜枫,顾庆,郝慧珍,等.基于内容的图像分割方法综述[J].软件学报,2017,28(1):160-183.DOI:10.13328/j.cnki.j0s.005136.JIANG F,GU Q,HAO H Z,et al.Survey on contentbased image segmentation methods[J].Journal of Software,2017,28(1):160-183.DOI:10.13328/j.cnki.j0s.005136(Ch).
[6]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2015:3431-3440.
[7]CHEN L C,PAPANDREOU G,et al.Deeplab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J].IEEETransactions on Pattern Analysis and Machine Intelligence,2018,40(4):834-848.DOI:10.1109/TPAMI.2017.2699184.
[8]ZHENG S,JAYASUMANA S,ROMERA-PAREDESB,et al.Conditional random fields as recurrent neural networks[C]//Proceedings of the IEEE International Conference on Computer Vision.Piscataway:IEEE,2015:1529-1537.DOI:10.1109/ICCV.2015.179.
[9]CHEN L C,YANG Y,WANG J,et al.Attention to scale:Scale-aware semantic image segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2016:3640-3649.
[10]LIN G,SHEN C,VAN DEN HENGEL A,et al.Efficient piecewise training of deep structured models for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2016:3194-3203.
[11]LIN T Y,MAIRE M,BELONGIE S,et al.Microsoft coco:Common objects in context[C]//Proceedings of the European Conference on Computer Vision.Berlin:Springer,2014:740-755.
[12]GOODFELLOW I,POUGET-ABADIE J,MIRZAM,et al.Generative adversarial nets[C]//Proceedings of the Advances in Neural Information Processing Systems.Massachusetts:MIT Press,2014:2672-2680.
[13]ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN[EB/OL].[2017-01-26].https://arxiv.org/pdf/1701.07875.pdf.
[14]ZHU J Y,PARK T,ISOLA P,et al.Unpaired Imageto-image Translation Using Cycle-Consistent Adversarial Networks[EB/OL].[2017-03-30].https://arxiv.org/pdf/1703.10593.pdf.
[15]MAO X D,LI Q,XIE H R,et al.Least squares generative adversarial networks[C]//Proceedings of the IEEEInternational Conference on Computer Vision.Piscataway:IEEE,2017:2813-2821.
[16]PATHAK D,KRAHENBUHL P,DONAHUE J,et al.Context encoders:Feature learning by inpainting[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2016:2536-2544.
[17]BADRINARAYANAN V,KENDALL A,CIPOLLA R.Segnet:A deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.DOI:10.1109/TPAMI.2016.2644615.
[18]HE K,ZHANG X,REN S,et al.Identity mappings in deep residual networks[C]//Proceedings of the European Conference on Computer Vision.Berlin:Springer,2016:630-645.
[19]CORDTS M,OMRAN M,RAMOS S,et al.The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2016:3213-3223.
[20]LIU M Y,TUZEL O.Coupled generative adversarial networks[C]//Proceedings of the Advances in Neural Information Processing Systems.Massachusetts:MIT Press,2016:469-477.
[21]DUMOULIN V,BELGHAZI I,POOLE B,et al.Adversarially Learned Inference[EB/OL].[2017-01-26].https://arxiv.org/pdf/1701.07875.pdf.
[22]SHRIVASTAVA A,PFISTER T,TUZEL O,et al.Learning from simulated and unsupervised images through adversarial training[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2017:2107-2116.
[2]施培蓓,刘贵全,汪中.基于快速增量学习的行人检测方法[J].小型微型计算机系统,2015,36(8):1837-1841.SHI P B,LIU G Q,WANG Z.Fast incremental learning method for pedestrian detection[J].Journal of Chinese Computer Systems,2015,36(8):1837-1841(Ch).
[3]徐岩,王权威,韦镇余.一种融合加权ELM和AdaBoost的交通标志识别算法[J].小型微型计算机系统,2017,38(9):2028-2032.DOI:10.3969/j.issn.1000-1220.2017.09.021.XU Y,WANG Q W,WEI Z Y.Traffic sign recognition algorithm combining weighted ELM and AdaBoost[J].Journal of Chinese Computer Systems,2017,38(9):2028-2032.DOI:10.3969/j.issn.1000-1220.2017.09.021.(Ch).
[4]魏云超,赵耀.基于DCNN的图像语义分割综述[J].北京交通大学学报,2016,40(4):82-91.DOI:10.11860/j.issn.1673-0291.2016.04.013.WEI Y C ZHAO Y.A review on image semantic segmentation based on DCNN[J].Journal of Beijing Jiaotong University,2016,40(4):82-91.DOI:10.11860/j.issn.1673-0291.2016.04.013(Ch).
[5]姜枫,顾庆,郝慧珍,等.基于内容的图像分割方法综述[J].软件学报,2017,28(1):160-183.DOI:10.13328/j.cnki.j0s.005136.JIANG F,GU Q,HAO H Z,et al.Survey on contentbased image segmentation methods[J].Journal of Software,2017,28(1):160-183.DOI:10.13328/j.cnki.j0s.005136(Ch).
[6]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2015:3431-3440.
[7]CHEN L C,PAPANDREOU G,et al.Deeplab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J].IEEETransactions on Pattern Analysis and Machine Intelligence,2018,40(4):834-848.DOI:10.1109/TPAMI.2017.2699184.
[8]ZHENG S,JAYASUMANA S,ROMERA-PAREDESB,et al.Conditional random fields as recurrent neural networks[C]//Proceedings of the IEEE International Conference on Computer Vision.Piscataway:IEEE,2015:1529-1537.DOI:10.1109/ICCV.2015.179.
[9]CHEN L C,YANG Y,WANG J,et al.Attention to scale:Scale-aware semantic image segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2016:3640-3649.
[10]LIN G,SHEN C,VAN DEN HENGEL A,et al.Efficient piecewise training of deep structured models for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2016:3194-3203.
[11]LIN T Y,MAIRE M,BELONGIE S,et al.Microsoft coco:Common objects in context[C]//Proceedings of the European Conference on Computer Vision.Berlin:Springer,2014:740-755.
[12]GOODFELLOW I,POUGET-ABADIE J,MIRZAM,et al.Generative adversarial nets[C]//Proceedings of the Advances in Neural Information Processing Systems.Massachusetts:MIT Press,2014:2672-2680.
[13]ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN[EB/OL].[2017-01-26].https://arxiv.org/pdf/1701.07875.pdf.
[14]ZHU J Y,PARK T,ISOLA P,et al.Unpaired Imageto-image Translation Using Cycle-Consistent Adversarial Networks[EB/OL].[2017-03-30].https://arxiv.org/pdf/1703.10593.pdf.
[15]MAO X D,LI Q,XIE H R,et al.Least squares generative adversarial networks[C]//Proceedings of the IEEEInternational Conference on Computer Vision.Piscataway:IEEE,2017:2813-2821.
[16]PATHAK D,KRAHENBUHL P,DONAHUE J,et al.Context encoders:Feature learning by inpainting[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2016:2536-2544.
[17]BADRINARAYANAN V,KENDALL A,CIPOLLA R.Segnet:A deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.DOI:10.1109/TPAMI.2016.2644615.
[18]HE K,ZHANG X,REN S,et al.Identity mappings in deep residual networks[C]//Proceedings of the European Conference on Computer Vision.Berlin:Springer,2016:630-645.
[19]CORDTS M,OMRAN M,RAMOS S,et al.The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2016:3213-3223.
[20]LIU M Y,TUZEL O.Coupled generative adversarial networks[C]//Proceedings of the Advances in Neural Information Processing Systems.Massachusetts:MIT Press,2016:469-477.
[21]DUMOULIN V,BELGHAZI I,POOLE B,et al.Adversarially Learned Inference[EB/OL].[2017-01-26].https://arxiv.org/pdf/1701.07875.pdf.
[22]SHRIVASTAVA A,PFISTER T,TUZEL O,et al.Learning from simulated and unsupervised images through adversarial training[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE,2017:2107-2116.