基于DDPG的无人车智能避障方法研究
|
摘要
本文中提出一种基于强化学习的无人车智能避障方法。鉴于无人车运动必须满足内外约束,包括汽车动力学约束和交通规则约束,且动作输出必须连续,而传统强化学习无法应对连续动作空间问题,提出了一种改进的DDPG算法,解决连续动作空间问题,实现转向盘转角和加速度的连续输出;采取多源传感器数据融合,满足无人车避障算法的状态输入;增加车辆内外约束条件,使输出动作更合理有效。最后,在开源仿真平台TORCS进行仿真,验证了算法的有效性和鲁棒性。
An intelligent obstacle avoidance scheme for unmanned vehicle based on reinforcement learning is proposed in this paper. In view of that the movement of unmanned vehicle must meet both interior and exterior constraints, including vehicle dynamics constraints and traffic rule constraints and its output must be continuous, which the traditional reinforcement learning cannot assure, an improved deep deterministic policy gradient algorithm is proposed to tackle continuous motion space issue and achieve the continuous output of steering wheel angle and acceleration. Multi-source sensor data fusion is adopted to fulfill the state input of unmanned vehicle obstacle avoidance algorithm and both interior and exterior constraints are added to make output motion more reasonable and effective. Finally a simulation is conducted on the open-source simulation platform TORCS and the effectiveness and robustness of the algorithm verified.
An intelligent obstacle avoidance scheme for unmanned vehicle based on reinforcement learning is proposed in this paper. In view of that the movement of unmanned vehicle must meet both interior and exterior constraints, including vehicle dynamics constraints and traffic rule constraints and its output must be continuous, which the traditional reinforcement learning cannot assure, an improved deep deterministic policy gradient algorithm is proposed to tackle continuous motion space issue and achieve the continuous output of steering wheel angle and acceleration. Multi-source sensor data fusion is adopted to fulfill the state input of unmanned vehicle obstacle avoidance algorithm and both interior and exterior constraints are added to make output motion more reasonable and effective. Finally a simulation is conducted on the open-source simulation platform TORCS and the effectiveness and robustness of the algorithm verified.
引文
[1] SALLAB A E, ABDOU M, PEROT E, et al. Deep reinforcement learning framework for autonomous driving[J]. Electronic Imaging,2017(19):70-76.
[2] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature,2015,518(7540):529-533.
[3] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[J]. Computer Science,2015,8(6):A187.
[4] XU Liangzheng, UIAO Chengyong, ZHANG J W. Analysis of dynamics stability of the tractor/full trailer combination vehicle and simulation for its control system[J]. Computer Simulation,2003,20(12):107-100.
[5] 王树凤,张大伟.车速与前轮转角的极限关系分析[J].机械设计与制造,2017(s1).
[6] 杨会会,宁丽娟.非线性漂移的Fokker-Planck方程的近似非定态解[J].物理学报,2013,62(18):38-45.
[7] XIONG X, WANG J, ZHANG F, et al. Combining deep reinforcement learning and safety based control for autonomous driving[J/OL]. https://arxiv.org/ftp/arxiv/papers/1612/1612.00147.pdf.
[2] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature,2015,518(7540):529-533.
[3] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[J]. Computer Science,2015,8(6):A187.
[4] XU Liangzheng, UIAO Chengyong, ZHANG J W. Analysis of dynamics stability of the tractor/full trailer combination vehicle and simulation for its control system[J]. Computer Simulation,2003,20(12):107-100.
[5] 王树凤,张大伟.车速与前轮转角的极限关系分析[J].机械设计与制造,2017(s1).
[6] 杨会会,宁丽娟.非线性漂移的Fokker-Planck方程的近似非定态解[J].物理学报,2013,62(18):38-45.
[7] XIONG X, WANG J, ZHANG F, et al. Combining deep reinforcement learning and safety based control for autonomous driving[J/OL]. https://arxiv.org/ftp/arxiv/papers/1612/1612.00147.pdf.