Adaptive dynamic programming for linear impulse systems

详细信息    查看全文

• 作者：Xiao-hua Wang ; Juan-juan Yu ; Yao Huang…
• 关键词：Adaptive dynamic programming (ADP) ; Impulse system ; Optimal control ; Neural network ; TP273.1
• 刊名：Frontiers of Information Technology & Electronic Engineering
• 出版年：2014
• 出版时间：January 2014
• 年：2014
• 卷：15
• 期：1
• 页码：43-50
• 全文大小：676 KB
• 参考文献：Ahmed, N.U., 2003. Existence of optimal controls for a general class of impulsive systems on Banach spaces. SIAM J. Control Optim., 42(2):669鈥?85. [doi:10.1137/S0363012901391299]CrossRef MATH MathSciNet
  Bainov, D.D., Simeonov, P.S., 1995. Impulsive Differential Equations: Asymptotic Properties of the Solutions. World Scientific, Singapore.MATH
  Balakrishnan, S.N., Ding, J., Lewis, F.L., 2008. Issues on stability of ADP feedback controllers for dynamical systems. IEEE Tran. Syst. Man Cybern. B, 38(4):913鈥?17. [doi:10.1109/TSMCB.2008.926599]CrossRef
  Bertsekas, D.P., 2011. Approximate policy iteration: a survey and some new methods. J. Control Theory Appl., 9(3):310鈥?35.CrossRef MATH MathSciNet
  Dierks, T., Jagannathan, S., 2011. Online optimal control of nonlinear discrete-time systems using approximate dynamic programming. J. Control Theory Appl., 9(3):361鈥?69. [doi:10.1007/s11768-011-0178-0]CrossRef MathSciNet
  Fraga, S.L., Pereira, F.L., 2012. Hamilton-Jacobi-Bellman equation and feedback synthesis for impulsive control. IEEE Trans. Automat. Control, 57(1):244鈥?49. [doi:10.1109/TAC.2011.2167822]CrossRef MathSciNet
  Jiang, Y., Jiang, Z.P., 2012. Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics. Automatica, 48(10):2699鈥?704. [doi:10.1016/j.automatica.2012.06.096]CrossRef MATH MathSciNet
  Jiang, Z.P., Jiang, Y., 2013. Robust adaptive dynamic programming for linear and nonlinear systems: an overview. Eur. J. Control, 19(5):417鈥?25. [doi:10.1016/j.ejcon.2013.05.017]CrossRef
  Kurzhanski, A.B., Daryin, A.N., 2008. Dynamic programming for impulse controls. Ann. Rev. Control, 32(2):213鈥?27. [doi:10.1016/j.arcontrol.2008.08.001]CrossRef
  Lakshmikantham, V., Bainov, D.D., Simeonov, P.S., 1989. Theory of Impulsive Differential Equations. World Scientific, Singapore.CrossRef MATH
  Lewis, F.L., Vrabie, D., 2009. Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circ. Syst. Mag., 9(3):32鈥?0. [doi:10.1109/MCAS.2009.933854]CrossRef MathSciNet
  Liu, B., Teo, K.L., Liu, X.Z., 2008. Optimal control and robust stability of uncertain impulsive dynamical systems. Asian J. Control, 10(3):314鈥?26. [doi:10.1002/asjc.37]CrossRef MathSciNet
  Liu, D.R., Wei, Q.L., 2013. Finite-approximation-errorbased optimal control approach for discrete-time nonlinear systems. IEEE Trans. Cybern., 43(2):779鈥?89. [doi:10.1109/TSMCB.2012.2216523]CrossRef
  Liu, X., 1995. Impulsive control and optimization. Appl. Math. Comput., 73(1):77鈥?8. [doi:10.1016/0096-3003(94)00204-H]CrossRef MATH MathSciNet
  Silva, G.N., Vinter, R.B., 1997. Necessary conditions for optimal impulsive control problems. SIAM J. Control Optim., 35(6):1829鈥?846. [doi:10.1137/S0363012995281857]CrossRef MATH MathSciNet
  Vamvoudakis, K.G., Lewis, F.L., 2010. Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica, 46(5):878鈥?88. [doi:10.1016/j.automatica.2010.02.018]CrossRef MATH MathSciNet
  Wang, F.Y., Zhang, H.G., Liu, D.R., 2009. Adaptive dynamic programming: an introduction. IEEE Comput. Intell. Mag., 4(2):39鈥?7. [doi:10.1109/MCI.2009.932261]CrossRef
  Wang, J.R., Yang, Y.L., 2010. Optimal control of linear impulsive antiperiodic boundary value problem on infinite dimensional spaces. Discr. Dynam. Nat. Soc., Article ID 673013. [doi:10.1155/2010/673013]
  Wang, X.H., 2008. Optimal Control of Impulsive Systems Using Adaptive Critic Neural Network. PhD Thesis, Missouri University of Science and Technology, Rolla, Missouri, USA.
  Wang, X.H., Balakrishnan, S.N., 2010. Optimal neurocontroller synthesis for impulse-driven systems. Neur. Networks, 23(1):125鈥?34. [doi:10.1016/j.neunet.2009.08.009]CrossRef MATH
  Wang, X.H., Luo, W.Z., Balakrishnan, S.N., 2012. Linear impulsive system optimization using adaptive dynamic programming. 12th Int. Conf. on Control Automation Robotics and Vision, p.725鈥?30. [doi:10.1109/ICARCV.2012.6485247]
  Werbos, P.J., 1974. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD Thesis, Harvard University, USA.
  Werbos, P.J., 2008. Foreword-ADP: the key direction for future research in intelligent control and understanding brain intelligence. IEEE Trans. Syst. Man Cybern. B, 38(4):898鈥?00. [doi:10.1109/TSMCB.2008.924139]CrossRef
  Werbos, P.J., McAvoy, T., Su, T., 1992. Handbook of Intelligent Control. Van Nostrand Reinhold, New York.
  Wu, Z., Zhang, F., 2011. Stochastic maximum principle for optimal control problems of forward-backward systems involving impulse controls. IEEE Trans. Automat. Control, 56(6):1401鈥?406.CrossRef MathSciNet
  Yang, T., 1999. Impulsive control. IEEE Trans. Automat. Control, 44(5):1081鈥?083.CrossRef MATH MathSciNet
  
• 作者单位：Xiao-hua Wang (1) (2)
  Juan-juan Yu (1)
  Yao Huang (1)
  Hua Wang (1) (2)
  Zhong-hua Miao (1) (2)
  
  1. School of Mechatronics Engineering and Automation, Shanghai University, Shanghai, 200072, China
  2. Shanghai Key Laboratory of Power Station Automation Technology, Shanghai University, Shanghai, 200072, China
  
• 刊物类别：Computer Science, general; Electrical Engineering; Computer Hardware; Computer Systems Organization
• 刊物主题：Computer Science, general; Electrical Engineering; Computer Hardware; Computer Systems Organization and Communication Networks; Electronics and Microelectronics, Instrumentation; Communications Engine
• 出版者：Zhejiang University Press
• ISSN：2095-9230

文摘

We investigate the optimization of linear impulse systems with the reinforcement learning based adaptive dynamic programming (ADP) method. For linear impulse systems, the optimal objective function is shown to be a quadric form of the pre-impulse states. The ADP method provides solutions that iteratively converge to the optimal objective function. If an initial guess of the pre-impulse objective function is selected as a quadratic form of the pre-impulse states, the objective function iteratively converges to the optimal one through ADP. Though direct use of the quadratic objective function of the states within the ADP method is theoretically possible, the numerical singularity problem may occur due to the matrix inversion therein when the system dimensionality increases. A neural network based ADP method can circumvent this problem. A neural network with polynomial activation functions is selected to approximate the pre-impulse objective function and trained iteratively using the ADP method to achieve optimal control. After a successful training, optimal impulse control can be derived. Simulations are presented for illustrative purposes. Key words Adaptive dynamic programming (ADP) Impulse system Optimal control Neural network