Reinforcement learning and optimal adaptive control: An overview and implementation examples

详细信息	查看全文 | 推荐本文 |

• 作者：Said G. Khan^a ; ^{mesgk@bris.ac.uk}  ; [Author Vitae] ; Guido Herrmann^b  ; [Author Vitae] ; Frank L. Lewis^c  ; [Author Vitae] ; Tony Pipe^a  ; [Author Vitae] ; Chris Melhuish^d  ; [Author Vitae]
• 关键词：Reinforcement learning ; ADP ; Q-learning ; Optimal adaptive control
• 刊名：Annual Reviews in Control
• 出版年：2012
• 期刊代码：11_13675788
• 类别：et
• 出版时间：April, 2012
• 卷：36
• 期：1
• 页码：42-59
• 文件大小：2201 K

摘要

This paper provides an overview of the reinforcement learning and optimal adaptive control literature and its application to robotics. Reinforcement learning is bridging the gap between traditional optimal control, adaptive control and bio-inspired learning techniques borrowed from animals. This work is highlighting some of the key techniques presented by well known researchers from the combined areas of reinforcement learning and optimal control theory. At the end, an example of an implementation of a novel model-free Q-learning based discrete optimal adaptive controller for a humanoid robot arm is presented. The controller uses a novel adaptive dynamic programming (ADP) reinforcement learning (RL) approach to develop an optimal policy on-line. The RL joint space tracking controller was implemented for two links (shoulder flexion and elbow flexion joints) of the arm of the humanoid Bristol-Elumotion-Robotic-Torso II (BERT II) torso. The constrained case (joint limits) of the RL scheme was tested for a single link (elbow flexion) of the BERT II arm by modifying the cost function to deal with the extra nonlinearity due to the joint constraints.