基于长短期记忆的车辆行为动态识别网络
|
摘要
高级辅助驾驶装置采用机器视觉技术实时处理摄录的行车前方车辆视频,动态识别并预估其姿态和行为。针对该类识别算法精度低、延迟大的问题,提出一种基于长短期记忆(LSTM)的车辆行为动态识别深度学习算法。首先,提取车辆行为视频中的关键帧;其次,引入双卷积网络并行对关键帧的特征信息进行分析,再利用LSTM网络对提取出的特性信息进行序列建模;最后,通过输出的预测得分判断出车辆行为类别。实验结果表明,所提算法识别准确率可达95.6%,对于单个视频的识别时间只要1.72 s;基于自建数据集,改进的双卷积算法相比普通卷积网络在准确率上提高8.02%,与传统车辆行为识别算法相比准确率提高6.36%。
In the advanced assisted driving device, machine vision technology was used to process the video of vehicles in front in real time to dynamically recognize and predict the posture and behavior of vehicle. Concerning low precision and large delay of this kind of recognition algorithm, a deep learning algorithm for vehicle behavior dynamic recognition based on Long Short-Term Memory(LSTM) was proposed. Firstly, the key frames in vehicle behavior video were extracted. Secondly, a dual convolutional network was introduced to analyze the feature information of key frames in parallel, and then LSTM network was used to sequence the extracted characteristic information. Finally, the output predicted score was used to determine the behavior type of vehicle. The experimental results show that the proposed algorithm has an accuracy of 95.6%, and the recognition time of a single video is only 1.72 s. The improved dual convolutional network algorithm improves the accuracy by 8.02% compared with ordinary convolutional network and increases by 6.36% compared with traditional vehicle behavior recognition algorithm based on a self-built dataset.
In the advanced assisted driving device, machine vision technology was used to process the video of vehicles in front in real time to dynamically recognize and predict the posture and behavior of vehicle. Concerning low precision and large delay of this kind of recognition algorithm, a deep learning algorithm for vehicle behavior dynamic recognition based on Long Short-Term Memory(LSTM) was proposed. Firstly, the key frames in vehicle behavior video were extracted. Secondly, a dual convolutional network was introduced to analyze the feature information of key frames in parallel, and then LSTM network was used to sequence the extracted characteristic information. Finally, the output predicted score was used to determine the behavior type of vehicle. The experimental results show that the proposed algorithm has an accuracy of 95.6%, and the recognition time of a single video is only 1.72 s. The improved dual convolutional network algorithm improves the accuracy by 8.02% compared with ordinary convolutional network and increases by 6.36% compared with traditional vehicle behavior recognition algorithm based on a self-built dataset.
引文
[1] 陈放.高级驾驶辅助系统ADAS浅谈[J].各界,2018(1):188-191.(CHEN F.A dissertation on advanced driver assistance system[J].All Circles,2018(1):188-191.)
[2] KASPER D,WEIDL G,DANG T,et al.Object-oriented Bayesian networks for detection of lane change maneuvers[J].IEEE Intelligent Transportation Systems Magazine,2012,4(3):19-31.
[3] GADEPALLY V,KRISHNAMURTHY A,OZGUNER U.A framework for estimating driver decisions near intersections [J].IEEE Transactions on Intelligent Transportation Systems,2014,15(2):637-646.
[4] 黄鑫,肖世德,宋波.监控视频中的车辆异常行为检测[J].计算机系统应用,2018,27(2):125-131.(HUANG X,XIAO S D,SONG B.Detection of vehicle's abnormal behaviors in surveillance video[J].Computer Systems and Applications,2018,27(2):125-131.)
[5] 黄慧玲,杨明,王春香,等.基于前方车辆行为识别的碰撞预警系统[J].华中科技大学学报(自然科学版),2015,43(s1):117-121.(HUANG H L,YANG M,WANG C X,et al.Collision warning system based on forward vehicle behavior recognition[J].Journal of Huazhong University of Science and Technology (Natural Science Edition),2015,43(s1):117-121.)
[6] DONAHUE J,HENDRICKS L A,ROHRBACH M,et al.Long-term recurrent convolutional networks for visual recognition and description[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2015:2625-2634.
[7] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[8] 殷昊,李寿山,贡正仙,等.基于多通道LSTM的不平衡情绪分类方法[J].中文信息学报,2018,32(1):139-145.(YIN H,LI S S,GONG Z X,et al.Imbalanced emotion classification based on multi-channel LSTM[J].Journal of Chinese Information Processing,2018,32(1):139-145.)
[9] 郑毅,李凤,张丽,等.基于长短时记忆网络的人体姿态检测方法[J].计算机应用,2018,38(6):1568-1574.(ZHENG Y,LI F,ZHANG L,et al.Pose detection and classification with LSTM network[J].Journal of Computer Applications,2018,38(6):1568-1574.)
[10] GRAVES A.Supervised Sequence Labelling with Recurrent Neural Networks[M].Berlin:Springer,2012:385.
[11] 曹晋其,蒋兴浩,孙锬锋.基于训练图CNN特征的视频人体动作识别算法[J].计算机工程,2017,43(11):234-238.(CAO J Q,JIANG X H,SUN T F.Video human action recognition algorithm based on trained image CNN features[J].Computer Engineering,2017,43(11):234-238.)
[12] SIMONYAN K,ZISSERMAN A.Two-stream convolutional net-works for action recognition in videos[C]// Proceedings of the 2014 International Conference on Neural Information Processing Systems.Montréal:[s.n.],2014:568-576.
[13] NG J.Y,MATTHEW H,VIJAYANARASIMHAN S,et al.Beyond short snippets:deep networks for video classification[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2015:4694-4702.
[14] CHEN H F,CHEN J,HU R M,et al.Action recognition with temporal scale-invariant deep learning framework[J].China Communications,2017,14(2):163-172.
[15] DENG J,DONG W,SOCHER R,et al.ImageNet:a large-scale hierarchical image database [C]// Proceedings of the 2009 IEEE International Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2009:248-255.
[16] HE K M,ZHANG X Y,REN S Q,et.al.Deep residual learning for image recognition [C]// Proceedings of the 2016 IEEE International Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2016:770-778.
[2] KASPER D,WEIDL G,DANG T,et al.Object-oriented Bayesian networks for detection of lane change maneuvers[J].IEEE Intelligent Transportation Systems Magazine,2012,4(3):19-31.
[3] GADEPALLY V,KRISHNAMURTHY A,OZGUNER U.A framework for estimating driver decisions near intersections [J].IEEE Transactions on Intelligent Transportation Systems,2014,15(2):637-646.
[4] 黄鑫,肖世德,宋波.监控视频中的车辆异常行为检测[J].计算机系统应用,2018,27(2):125-131.(HUANG X,XIAO S D,SONG B.Detection of vehicle's abnormal behaviors in surveillance video[J].Computer Systems and Applications,2018,27(2):125-131.)
[5] 黄慧玲,杨明,王春香,等.基于前方车辆行为识别的碰撞预警系统[J].华中科技大学学报(自然科学版),2015,43(s1):117-121.(HUANG H L,YANG M,WANG C X,et al.Collision warning system based on forward vehicle behavior recognition[J].Journal of Huazhong University of Science and Technology (Natural Science Edition),2015,43(s1):117-121.)
[6] DONAHUE J,HENDRICKS L A,ROHRBACH M,et al.Long-term recurrent convolutional networks for visual recognition and description[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2015:2625-2634.
[7] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[8] 殷昊,李寿山,贡正仙,等.基于多通道LSTM的不平衡情绪分类方法[J].中文信息学报,2018,32(1):139-145.(YIN H,LI S S,GONG Z X,et al.Imbalanced emotion classification based on multi-channel LSTM[J].Journal of Chinese Information Processing,2018,32(1):139-145.)
[9] 郑毅,李凤,张丽,等.基于长短时记忆网络的人体姿态检测方法[J].计算机应用,2018,38(6):1568-1574.(ZHENG Y,LI F,ZHANG L,et al.Pose detection and classification with LSTM network[J].Journal of Computer Applications,2018,38(6):1568-1574.)
[10] GRAVES A.Supervised Sequence Labelling with Recurrent Neural Networks[M].Berlin:Springer,2012:385.
[11] 曹晋其,蒋兴浩,孙锬锋.基于训练图CNN特征的视频人体动作识别算法[J].计算机工程,2017,43(11):234-238.(CAO J Q,JIANG X H,SUN T F.Video human action recognition algorithm based on trained image CNN features[J].Computer Engineering,2017,43(11):234-238.)
[12] SIMONYAN K,ZISSERMAN A.Two-stream convolutional net-works for action recognition in videos[C]// Proceedings of the 2014 International Conference on Neural Information Processing Systems.Montréal:[s.n.],2014:568-576.
[13] NG J.Y,MATTHEW H,VIJAYANARASIMHAN S,et al.Beyond short snippets:deep networks for video classification[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2015:4694-4702.
[14] CHEN H F,CHEN J,HU R M,et al.Action recognition with temporal scale-invariant deep learning framework[J].China Communications,2017,14(2):163-172.
[15] DENG J,DONG W,SOCHER R,et al.ImageNet:a large-scale hierarchical image database [C]// Proceedings of the 2009 IEEE International Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2009:248-255.
[16] HE K M,ZHANG X Y,REN S Q,et.al.Deep residual learning for image recognition [C]// Proceedings of the 2016 IEEE International Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2016:770-778.