基于强化学习的旋翼无人机智能追踪方法
|
摘要
针对旋翼无人机追踪场景中常用的PID控制方法与视觉伺服控制方法的不足,该文尝试将视觉伺服控制与强化学习结合,提出了一种基于强化学习的旋翼无人机智能追踪方法。首先使用基于图像的视觉伺服实现旋翼无人机的闭环控制,然后建立使用Sarsa学习算法调节伺服增益的强化学习模型,通过训练可以使得旋翼无人机自主选择视觉伺服增益。该文设计了旋翼无人机在实物场景与仿真场景下的运动目标追踪实验,实验结果论证了该方法相对于PID控制与基于图像的视觉伺服控制方法具有更好的追踪效果。
Aiming at the deficiencies of PID control method and visual servo control method commonly used in the tracking scene of Rotor UAV(unmanned aerial vehicle), this paper attempts to combine visual servo control with reinforcement learning, and proposes an intelligent tracking method for Rotor UAV based on reinforcement learning. Firstly, image-based visual servo is used to track the closed-loop control of the Rotor UAV, and then a reinforcement learning model is established to adjust the servo gain with Sarsa learning algorithm. After many training sessions, the Rotor UAV can choose its own visual servo gain. In this paper, the experiment of tracking the moving target of Rotor UAV in physical and simulation scenarios is designed. The experimental results demonstrate that the proposed method has better tracking effect than PID control and classical image-based visual servo control method.
Aiming at the deficiencies of PID control method and visual servo control method commonly used in the tracking scene of Rotor UAV(unmanned aerial vehicle), this paper attempts to combine visual servo control with reinforcement learning, and proposes an intelligent tracking method for Rotor UAV based on reinforcement learning. Firstly, image-based visual servo is used to track the closed-loop control of the Rotor UAV, and then a reinforcement learning model is established to adjust the servo gain with Sarsa learning algorithm. After many training sessions, the Rotor UAV can choose its own visual servo gain. In this paper, the experiment of tracking the moving target of Rotor UAV in physical and simulation scenarios is designed. The experimental results demonstrate that the proposed method has better tracking effect than PID control and classical image-based visual servo control method.
引文
[1]CHOI S,KIM S,KIM H J.Inverse reinforcement learning control for trajectory tracking of a multirotor UAV[J].International Journal of Control Automation&Systems,2017,15(4):1826-1834.
[2]LIU Y,MONTENBRUCK J M,ZELAZO D,et al.Adistributed control approach to formation balancing and maneuvering of multiple multirotor UAVs[J].IEEETransactions on Robotics,2018,34(4):870-882.
[3]SI W,SHE H,WANG Z.Fuzzy PID controller for UAVtracking moving target[C]//Control and Decision Conference.Chongqing,China:IEEE,2017:3023-3027.
[4]TEULIERE C,MARCHAND E,ECK L.3-D Model-based tracking for UAV indoor localization[J].IEEE Transactions on Cybernetics,2015,45(5):869-879.
[5]ZHENG D,WANG H,CHEN W,et al.Planning and tracking in image space for image-based visual servoing of a quadrotor[J].IEEE Transactions on Industrial Electronics,2018,65(4):3376-3385.
[6]WANG Y,ZHANG G L,LANG H,et al.A modified image-based visual servo controller with hybrid camera configuration for robust robotic grasping[J].Robotics&Autonomous Systems,2014,62(10):1398-1407.
[7]CHAUMETTE F,HUTCHINSON S.Visual servo control,part I:Basic approaches[J].IEEE Robotics&Automation Magazine,2009,13(4):82-90.
[8]CHO S,LEE D,SHIM D H.Image-based visual servoing framework for a multirotor UAV using sampling-based path planning[C]//AIAA Guidance,Navigation,&Control Conference.Kissimmee,Floride:AIAA,2015:1-15.
[9]SERRA P,CUNHA R,HAMEL T,et al.Landing of a quadrotor on a moving target using dynamic image-based visual servo control[J].IEEE Transactions on Robotics,2016,32(6):1524-1535.
[10]LEE D,KIM S J.Modified chain-code-based object recognition[J].Electronics Letters,2015,51(24):1996-1997.
[11]ZHENG N,MA Q,JIN M,et al.Abdominal-waving control of tethered bumblebees based on Sarsa with transformed reward[J].IEEE Transactions on Cybernetics,2018,49(8):3064-3073.
[12]LI J,CHAI T,LEWIS F,et al.Off-policy Q-learning:Set-point design for optimizing dual-rate rougher flotation operational processes[J].IEEE Transactions on Industrial Electronics,2017,65(5):4092-4102.
[2]LIU Y,MONTENBRUCK J M,ZELAZO D,et al.Adistributed control approach to formation balancing and maneuvering of multiple multirotor UAVs[J].IEEETransactions on Robotics,2018,34(4):870-882.
[3]SI W,SHE H,WANG Z.Fuzzy PID controller for UAVtracking moving target[C]//Control and Decision Conference.Chongqing,China:IEEE,2017:3023-3027.
[4]TEULIERE C,MARCHAND E,ECK L.3-D Model-based tracking for UAV indoor localization[J].IEEE Transactions on Cybernetics,2015,45(5):869-879.
[5]ZHENG D,WANG H,CHEN W,et al.Planning and tracking in image space for image-based visual servoing of a quadrotor[J].IEEE Transactions on Industrial Electronics,2018,65(4):3376-3385.
[6]WANG Y,ZHANG G L,LANG H,et al.A modified image-based visual servo controller with hybrid camera configuration for robust robotic grasping[J].Robotics&Autonomous Systems,2014,62(10):1398-1407.
[7]CHAUMETTE F,HUTCHINSON S.Visual servo control,part I:Basic approaches[J].IEEE Robotics&Automation Magazine,2009,13(4):82-90.
[8]CHO S,LEE D,SHIM D H.Image-based visual servoing framework for a multirotor UAV using sampling-based path planning[C]//AIAA Guidance,Navigation,&Control Conference.Kissimmee,Floride:AIAA,2015:1-15.
[9]SERRA P,CUNHA R,HAMEL T,et al.Landing of a quadrotor on a moving target using dynamic image-based visual servo control[J].IEEE Transactions on Robotics,2016,32(6):1524-1535.
[10]LEE D,KIM S J.Modified chain-code-based object recognition[J].Electronics Letters,2015,51(24):1996-1997.
[11]ZHENG N,MA Q,JIN M,et al.Abdominal-waving control of tethered bumblebees based on Sarsa with transformed reward[J].IEEE Transactions on Cybernetics,2018,49(8):3064-3073.
[12]LI J,CHAI T,LEWIS F,et al.Off-policy Q-learning:Set-point design for optimizing dual-rate rougher flotation operational processes[J].IEEE Transactions on Industrial Electronics,2017,65(5):4092-4102.