结合KSW和FCNN的道路场景分割
|
摘要
目的随着自动驾驶技术不断引入生活,机器视觉中道路场景分割算法的研究已至关重要。传统方法中大多数研究者使用机器学习方法对阈值分割,而近年来深度学习的引入,使得卷积神经网络被广泛应用于该领域。方法针对传统阈值分割方法难以有效提取多场景下道路图像阈值的问题和直接用深度神经网络来训练数据导致过分割严重的问题,本文提出了结合KSW(key seat wiper)和全卷积神经网络(FCNN)的道路场景分割方法,该方法结合了KSW熵法及遗传算法,利用深度学习在不同场景下的特征提取,并将其运用到无人驾驶技术的道路分割中。首先对道路场景测试集利用KSW熵法及遗传算法得到训练集,然后导入到全卷积神经网络中进行训练得到有效训练模型,最后通过训练模型实现对任意一幅道路场景图分割。结果实验结果表明,在KITTI数据集中进行测试,天空和树木的分割精度分别达到91. 3%和94. 3%,道路、车辆、行人的分割精度提高了2%左右。从分割结果中明显看出,道路图像中的积水、泥潭、树木等信息存在的过分割现象有良好的改观。结论相比传统机器学习道路场景分割方法,本文方法在一定程度上提高了分割精度。对比深度学习直接应用于道路场景分割的方法,本文方法在一定程度上避免了过分割现象,提高了模型的鲁棒性。综上所述,本文提出的结合KSW和FCNN的道路场景分割算法有广泛的研究前景,有望应用于医学图像和遥感图像的处理中。
Objective The advent of driverless cars has become a hot topic in todays' society. Driverless aims to achieve a high degree of autonomous driving behavior through environmental awareness,such as in starting,braking,lane line tracking,lane changing,collision avoidance,and parking. Image segmentation of road scenes plays an important role in this technology. Studying the manner in which complex scenes and high-efficiency scene segmentation images in the environment of severe noise interference are achieved is essential. Traditional road segmentation generally uses a binocular stereo vision map and a motion indicator-based approach. For example,some researchers proposed a pedestrian detection based on binocular stereo vision and SVM( support vector machine) algorithm and used threshold segmentation to determine the coordinate position of a moving target. For the diversity of motion indicators,other researchers used the projection surface direction and object. Multiple motion indicators,such as altitude and feature tracking density,segment the road. However,these methods have high requirements on computing resources. For the current unmanned practicality requirements,a concise and resource-intensive method is required. Since 2012,deep learning has been gradually introduced into road scene segmentation. A scholar proposed a smart car steering study based on end-to-end depth learning and obtained good road feature coding through pretraining self-encoding. In recent years,AI technology has suddenly caught the interest of scholars.Computer GPU parallel operation,computational acceleration,storage space compression,and other technologies are studied and developed. Large amount of data and calculation are no longer restricted. Convolutional neural network( CNN) has become a research hotspot and has been widely used. Some researchers studied the deep learning algorithm of CNN to learn high-order features in a scene to achieve road scene segmentation. However,to some extent,although the computational strength is reduced,some problems of over-segmentation of complex scenes. Other researchers proposed the feature automatic extraction capability of deep structure using deep CNN for complex scene problems; it is a method of feature selfencoder versus feature similarity metric in source-target scenarios. However,these algorithms do not achieve the desired results for road marking,vehicle,and pedestrian segmentation accuracy. During rainy days,snowy days,and high-temperature weather,road surface often appears to be divided. With the continuous introduction of autonomous driving technology,the study of road scene segmentation algorithms in machine vision has become crucial. Most researchers in the traditional methods use machine learning to segment thresholds. The introduction of deep learning in recent years has caused the wide usage of neural network in this field. Method A road scene combining KSW( key seat wiper) and full CNN( FCNN) is proposed to address the problem of the traditional threshold segmentation method in terms of difficulty in extracting the road image threshold under multiple scenes and the training of data directly by deep neural network causing over-segmentation.The segmentation method,which combines the KSW entropy method and genetic algorithm,uses depth learning to extract features in different scenarios and applies it to the road segmentation of unmanned technology. First,the original test image of the road scene is converted into gray image,and the filtering effect is achieved by KSW genetic algorithm two-dimensional threshold segmentation; thus,the road water,road standard line,trees,and other scenes are clear in the image,and the preprocessing training set is obtained. Import pre-trained datasets into the FCNN framework After tens of thousands of training iterations,it learns and amends the weights,thereby resulting in an effective training model. Finally,the training model can be used to implement any road scene graph segmentation. Result Experimental results show that the segmentation accuracy of the sky and trees reached 91. 3% and 94. 3% in the KITTI dataset,respectively,and the segmentation progress of roads,vehicles,and pedestrians increased by approximately 2%. In comparison with the previous super parsing and boosting algorithms,the proposed algorithm can distinguish road segmentation lines and increase the segmentation accuracy of trees,vehicles,and pedestrians by approximately 20%. Comparing with the result of Seg Net and Res Net depth networks,in the environment of roads and trees. The segmentation accuracy of road segmentation lines is relatively improved by approximately 5%. In comparison with the original image,the KSW two-dimensional threshold and genetic algorithm filter out the excessively bright part of the sun on the road and the overly bright part of the sky to prevent over-segmentation of the segmented image. Conclusion Segmentation result shows that the over-segmentation of water accumulation and mud on the road has been remarkably improved. In comparison with the traditional machine learning road scene segmentation method,the proposed method improves the segmentation accuracy to a certain extent. In comparison with the depth learning method,the proposed method is directly applied to the road scene segmentation. This method avoids the over-segmentation phenomenon to some extent and improves the model robustness. In summary,the proposed road scene segmentation algorithm combined with KSW and FCNN has broad research prospects and is expected to be applied to the processing of medical and remote sensing images.
Objective The advent of driverless cars has become a hot topic in todays' society. Driverless aims to achieve a high degree of autonomous driving behavior through environmental awareness,such as in starting,braking,lane line tracking,lane changing,collision avoidance,and parking. Image segmentation of road scenes plays an important role in this technology. Studying the manner in which complex scenes and high-efficiency scene segmentation images in the environment of severe noise interference are achieved is essential. Traditional road segmentation generally uses a binocular stereo vision map and a motion indicator-based approach. For example,some researchers proposed a pedestrian detection based on binocular stereo vision and SVM( support vector machine) algorithm and used threshold segmentation to determine the coordinate position of a moving target. For the diversity of motion indicators,other researchers used the projection surface direction and object. Multiple motion indicators,such as altitude and feature tracking density,segment the road. However,these methods have high requirements on computing resources. For the current unmanned practicality requirements,a concise and resource-intensive method is required. Since 2012,deep learning has been gradually introduced into road scene segmentation. A scholar proposed a smart car steering study based on end-to-end depth learning and obtained good road feature coding through pretraining self-encoding. In recent years,AI technology has suddenly caught the interest of scholars.Computer GPU parallel operation,computational acceleration,storage space compression,and other technologies are studied and developed. Large amount of data and calculation are no longer restricted. Convolutional neural network( CNN) has become a research hotspot and has been widely used. Some researchers studied the deep learning algorithm of CNN to learn high-order features in a scene to achieve road scene segmentation. However,to some extent,although the computational strength is reduced,some problems of over-segmentation of complex scenes. Other researchers proposed the feature automatic extraction capability of deep structure using deep CNN for complex scene problems; it is a method of feature selfencoder versus feature similarity metric in source-target scenarios. However,these algorithms do not achieve the desired results for road marking,vehicle,and pedestrian segmentation accuracy. During rainy days,snowy days,and high-temperature weather,road surface often appears to be divided. With the continuous introduction of autonomous driving technology,the study of road scene segmentation algorithms in machine vision has become crucial. Most researchers in the traditional methods use machine learning to segment thresholds. The introduction of deep learning in recent years has caused the wide usage of neural network in this field. Method A road scene combining KSW( key seat wiper) and full CNN( FCNN) is proposed to address the problem of the traditional threshold segmentation method in terms of difficulty in extracting the road image threshold under multiple scenes and the training of data directly by deep neural network causing over-segmentation.The segmentation method,which combines the KSW entropy method and genetic algorithm,uses depth learning to extract features in different scenarios and applies it to the road segmentation of unmanned technology. First,the original test image of the road scene is converted into gray image,and the filtering effect is achieved by KSW genetic algorithm two-dimensional threshold segmentation; thus,the road water,road standard line,trees,and other scenes are clear in the image,and the preprocessing training set is obtained. Import pre-trained datasets into the FCNN framework After tens of thousands of training iterations,it learns and amends the weights,thereby resulting in an effective training model. Finally,the training model can be used to implement any road scene graph segmentation. Result Experimental results show that the segmentation accuracy of the sky and trees reached 91. 3% and 94. 3% in the KITTI dataset,respectively,and the segmentation progress of roads,vehicles,and pedestrians increased by approximately 2%. In comparison with the previous super parsing and boosting algorithms,the proposed algorithm can distinguish road segmentation lines and increase the segmentation accuracy of trees,vehicles,and pedestrians by approximately 20%. Comparing with the result of Seg Net and Res Net depth networks,in the environment of roads and trees. The segmentation accuracy of road segmentation lines is relatively improved by approximately 5%. In comparison with the original image,the KSW two-dimensional threshold and genetic algorithm filter out the excessively bright part of the sun on the road and the overly bright part of the sky to prevent over-segmentation of the segmented image. Conclusion Segmentation result shows that the over-segmentation of water accumulation and mud on the road has been remarkably improved. In comparison with the traditional machine learning road scene segmentation method,the proposed method improves the segmentation accuracy to a certain extent. In comparison with the depth learning method,the proposed method is directly applied to the road scene segmentation. This method avoids the over-segmentation phenomenon to some extent and improves the model robustness. In summary,the proposed road scene segmentation algorithm combined with KSW and FCNN has broad research prospects and is expected to be applied to the processing of medical and remote sensing images.
引文
[1]Chen S Y,Wang H H,Wu Y T,et al.Approach of pedestrian detection based on binocular stereo vision and SVM algorithm[J].Journal of Huazhong University of Science and Technology:Nature Science Edition,2015,43(S1):140-143.[陈双玉,王海晖,吴云韬,等.基于双目立体视觉和SVM算法行人检测方法[J].华中科技大学学报:自然科学版,2015,43(S1):140-143.][DOI:10.13245/j.hust.15S1034]
[2]Sturgess P,Alahari K,Ladicky L,et al.Combining appearance and structure from motion features for road scene understanding[C]//British Machine Vision Conference.London,UK:BMVAPress,2009:1-10.
[3]Zou B,Li C Q,Hou X J,et al.Research on automatic steering of intelligent vehicle based on end-to-end deep learning[J].Application Research of Computers,2018,35(9):2873-2876.[邹斌,李超群,侯献军,等.基于端到端深度学习的智能车自动转向研究[J].计算机应用研究,2018,35(9):2873-2876.][DOI:10.3969/j.issn.1001-3695.2018.09.074]
[4]Alvarez J M,Lecun Y,Gevers T,et al.Semantic road segmentation via multi-scale ensembles of learned features[C]//European Conference on Computer Vision-ECCV 2012.Florence,Italy:Springer,2012:586-595.[DOI:10.1007/978-3-642-33868-7_58]
[5]Wang H,Cai Y F,Jia Y Y,et al.Scene adaptive road segmentation algorithm based on deep convolutional neural network[J].Journal of Electronics&Information Technology,2017,39(2):263-269.[王海,蔡英凤,贾允毅,等.基于深度卷积神经网络的场景自适应道路分割算法[J].电子与信息学报,2017,39(2):263-269.][DOI:10.11999/JEIT160329]
[6]Cao F Y,Hu Y J,Wang H,et al.Image object segmentation algorithm by T junctions information[J].Journal of Image and Graphics,2018,23(3):428-433.[曹风云,胡玉娟,王浩,等.融合T节点线索的图像物体分割[J].中国图象图形学报,2018,23(3):428-433.][DOI:10.11834/jig.170298]
[7]Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE,2015:3431-3440.[DOI:10.1109/CVPR.2015.7298965]
[8]Krizhevsky A,Sutskever I,Hinton G E.Image Net classification with deep convolutional neural networks[C]//Proceedings of the25th International Conference on Neural Information Processing Systems.Lake Tahoe,Nevada:ACM,2012:1097-1105.
[9]Liu Y P,Cai W L,Hong G B,et al.Automatic segmentation of shoulder joint in MRI by using patch-wise and full-image fully convolutional networks[J].Journal of Image and Graphics,2018,23(10):1558-1570.[刘云鹏,蔡文立,洪国斌,等.应用图像块和全卷积神经网络的肩关节MRI自动分割[J].中国图象图形学报,2018,23(10):1558-1570.][DOI:10.11834/jig.180044]
[10]Tuo Q.Research on Image Threshold Segmentation Algorithm Based on Maximum Entropy and Genetic Algorithm[D].Kunming:Kunming University of Science and Technology,2016.[庹谦.最大熵结合遗传算法的图像阈值分割算法研究[D].昆明:昆明理工大学,2016.]
[11]Sun X F,Peng T Q.Fully convolutional networks for instance search[J].Journal of Henan Institute of Engineering,2017,29(3):65-71.[孙晓峰,彭天强.一种基于全卷积网络的目标检索方法[J].河南工程学院学报:自然科学版,2017,29(3):65-71.][DOI:10.3969/j.issn.1674-330X.2017.03.015]
[12]Gao K J,Sun S Y,Yao G S,et al.Semantic segmentation of night vision images for unmanned vehicles based on deep learning[J].Journal of Applied Optics,2017,38(3):421-428.[高凯珺,孙韶媛,姚广顺,等.基于深度学习的无人车夜视图像语义分割[J].应用光学,2017,38(3):421-428.][DOI:10.5768/JAO201738.0302007]
[13]Zhang J.Image Semantic Segmentation Method Based on Deep Learning[D].University of Electronic Science and Technology,2018.[张建.基于深度学习的图像语义分割方法[D].电子科技大学,2018.]
[2]Sturgess P,Alahari K,Ladicky L,et al.Combining appearance and structure from motion features for road scene understanding[C]//British Machine Vision Conference.London,UK:BMVAPress,2009:1-10.
[3]Zou B,Li C Q,Hou X J,et al.Research on automatic steering of intelligent vehicle based on end-to-end deep learning[J].Application Research of Computers,2018,35(9):2873-2876.[邹斌,李超群,侯献军,等.基于端到端深度学习的智能车自动转向研究[J].计算机应用研究,2018,35(9):2873-2876.][DOI:10.3969/j.issn.1001-3695.2018.09.074]
[4]Alvarez J M,Lecun Y,Gevers T,et al.Semantic road segmentation via multi-scale ensembles of learned features[C]//European Conference on Computer Vision-ECCV 2012.Florence,Italy:Springer,2012:586-595.[DOI:10.1007/978-3-642-33868-7_58]
[5]Wang H,Cai Y F,Jia Y Y,et al.Scene adaptive road segmentation algorithm based on deep convolutional neural network[J].Journal of Electronics&Information Technology,2017,39(2):263-269.[王海,蔡英凤,贾允毅,等.基于深度卷积神经网络的场景自适应道路分割算法[J].电子与信息学报,2017,39(2):263-269.][DOI:10.11999/JEIT160329]
[6]Cao F Y,Hu Y J,Wang H,et al.Image object segmentation algorithm by T junctions information[J].Journal of Image and Graphics,2018,23(3):428-433.[曹风云,胡玉娟,王浩,等.融合T节点线索的图像物体分割[J].中国图象图形学报,2018,23(3):428-433.][DOI:10.11834/jig.170298]
[7]Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE,2015:3431-3440.[DOI:10.1109/CVPR.2015.7298965]
[8]Krizhevsky A,Sutskever I,Hinton G E.Image Net classification with deep convolutional neural networks[C]//Proceedings of the25th International Conference on Neural Information Processing Systems.Lake Tahoe,Nevada:ACM,2012:1097-1105.
[9]Liu Y P,Cai W L,Hong G B,et al.Automatic segmentation of shoulder joint in MRI by using patch-wise and full-image fully convolutional networks[J].Journal of Image and Graphics,2018,23(10):1558-1570.[刘云鹏,蔡文立,洪国斌,等.应用图像块和全卷积神经网络的肩关节MRI自动分割[J].中国图象图形学报,2018,23(10):1558-1570.][DOI:10.11834/jig.180044]
[10]Tuo Q.Research on Image Threshold Segmentation Algorithm Based on Maximum Entropy and Genetic Algorithm[D].Kunming:Kunming University of Science and Technology,2016.[庹谦.最大熵结合遗传算法的图像阈值分割算法研究[D].昆明:昆明理工大学,2016.]
[11]Sun X F,Peng T Q.Fully convolutional networks for instance search[J].Journal of Henan Institute of Engineering,2017,29(3):65-71.[孙晓峰,彭天强.一种基于全卷积网络的目标检索方法[J].河南工程学院学报:自然科学版,2017,29(3):65-71.][DOI:10.3969/j.issn.1674-330X.2017.03.015]
[12]Gao K J,Sun S Y,Yao G S,et al.Semantic segmentation of night vision images for unmanned vehicles based on deep learning[J].Journal of Applied Optics,2017,38(3):421-428.[高凯珺,孙韶媛,姚广顺,等.基于深度学习的无人车夜视图像语义分割[J].应用光学,2017,38(3):421-428.][DOI:10.5768/JAO201738.0302007]
[13]Zhang J.Image Semantic Segmentation Method Based on Deep Learning[D].University of Electronic Science and Technology,2018.[张建.基于深度学习的图像语义分割方法[D].电子科技大学,2018.]