A neural-network-based online optimal control approach for nonlinear robust decentralized stabilization

详细信息    查看全文

• 作者：Ding Wang ; Derong Liu ; Hongliang Li ; Hongwen Ma ; Chao Li
• 关键词：Adaptive dynamic programming ; Approximate dynamic programming ; Neural networks ; Online optimal control ; Robust decentralized stabilization ; Uncertain nonlinear systems
• 刊名：Soft Computing - A Fusion of Foundations, Methodologies and Applications
• 出版年：2016
• 出版时间：February 2016
• 年：2016
• 卷：20
• 期：2
• 页码：707-716
• 全文大小：837 KB
• 参考文献：Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5):779–791MATH MathSciNet CrossRef
  Adhyaru DM, Kar IN, Gopal M (2011) Bounded robust control of nonlinear systems using neural network-based HJB solution. Neural Comput Appl 20(1):91–103CrossRef
  Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern Part B Cybern 38(4):943–949CrossRef
  Bhasin S, Kamalapurkar R, Johnson M, Vamvoudakis KG, Lewis FL, Dixon WE (2013) A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems. Automatica 49(1):82–92MATH MathSciNet CrossRef
  Dierks T, Jagannathan S (2012) Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update. IEEE Trans Neural Netw Learn Syst 23(7):1118–1129CrossRef
  Dierks T, Jagannathan S (2010) Optimal control of affine nonlinear continuous-time systems. In: Proceedings of the American control conference, Baltimore, MD, USA, June 2010, pp 1568–1573
  Haddad WM, Chellaboina V, Fausz JL (1998) Robust nonlinear feedback control for uncertain linear systems with nonquadratic performance criteria. Syst Control Lett 33(5):327–338MATH MathSciNet CrossRef
  Haddad WM, Chellaboina V, Fausz JL (2000) Optimal non-linear robust control for non-linear uncertain systems. Int J Control 73(4):329–342MATH MathSciNet CrossRef
  Heydari A, Balakrishnan SN (2013) Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics. IEEE Trans Neural Netw Learn Syst 24(1):145–157CrossRef
  Jiang ZP, Jiang Y (2013) Robust adaptive dynamic programming for linear and nonlinear systems: an overview. Eur J Control 19(5):417–425MATH MathSciNet CrossRef
  Lewis FL, Jagannathan S, Yesildirek A (1999) Neural network control of robot manipulators and nonlinear systems. Taylor & Francis, London
  Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers. IEEE Control Syst Mag 32(6):76–105MathSciNet CrossRef
  Lin F (2000) An optimal control approach to robust control design. Int J Control 73(3):177–186MATH CrossRef
  Liu D, Wang D, Zhao D, Wei Q, Jin N (2012) Neural-network-based optimal control for a class of unknown discrete-time nonlinear systems using globalized dual heuristic programming. IEEE Trans Autom Sci Eng 9(3):628–634CrossRef
  Liu D, Wang D, Yang X (2013a) An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs. Inf Sci 220:331–342
  Liu D, Li H, Wang D (2013b) Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm. Neurocomputing 110:92–100
  Liu D, Li H, Wang D (2013c) Data-based self-learning optimal control: research progress and prospects. Acta Automatica Sinica 39(11):1858–1870
  Liu D, Wang D, Li H (2014a) Decentralized stabilization for a class of continuous-time nonlinear interconnected systems using online learning optimal control approach. IEEE Trans Neural Netw Learn Syst 25(2):418–428
  Liu D, Li H, Wang D (2014b) Online synchronous approximate optimal learning algorithm for multiplayer nonzero-sum games with unknown dynamics. IEEE Trans Syst Man Cybern Syst 44(8):1015–1027
  Ni Z, He H, Wen J (2013) Adaptive learning in tracking control based on the dual critic network design. IEEE Trans Neural Netw Learn Syst 24(6):913–928CrossRef
  Ni Z, He H (2013) Heuristic dynamic programming with internal goal representation. Soft Comput 17(11):2101–2108CrossRef
  Nodland D, Zargarzadeh H, Jagannathan S (2013) Neural network-based optimal adaptive output feedback control of a helicopter UAV. IEEE Trans Neural Netw Learn Syst 24(7):1061–1073
  Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888
  Wang D, Liu D, Wei Q (2012a) Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach. Neurocomputing 78(1):14–22
  Wang D, Liu D, Wei Q, Zhao D, Jin N (2012b) Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica 48(8):1825–1832
  Wang D, Liu D, Li H (2014) Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems. IEEE Trans Autom Sci Eng 11(2):627–632CrossRef
  Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Proceedings of handbook of intelligent control: neural, fuzzy, and adaptive approaches, ch 13, Van Nostrand Reinhold, New York
  Wu HN, Luo B (2012) Neural network based online simultaneous policy update algorithm for solving the HJI equation in nonlinear \(H_{\infty }\) control. IEEE Trans Neural Netw Learn Syst 23(12):1884–1895MathSciNet CrossRef
  Yang X, Liu D, Wang D (2014) Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints. Int J Control 87(3):553–566MathSciNet CrossRef
  Zhang H, Luo Y, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20(9):1490–1503CrossRef
  Zhang H, Cui L, Luo Y (2013) Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network ADP. IEEE Trans Cybern 43(1):206–216CrossRef
  Zhang D, Liu D, Wang D (2014) Approximate optimal solution of the DTHJB equation for a class of nonlinear affine systems with unknown dead-zone constraints. Soft Comput 18(2):349–357CrossRef
  Zhao Q, Xu H, Dierks T, Jagannathan S (2013) Finite-horizon neural network-based optimal control design for affine nonlinear continuous-time systems. In: Proceedings of the international joint conference on neural networks, Dallas, TX, USA, Aug 2013, pp 1–6
  
• 作者单位：Ding Wang (1)
  Derong Liu (1)
  Hongliang Li (1)
  Hongwen Ma (1)
  Chao Li (1)
  
  1. The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China
  
• 刊物类别：Engineering
• 刊物主题：Numerical and Computational Methods in Engineering
  Theory of Computation
  Computing Methodologies
  Mathematical Logic and Foundations
  Control Engineering
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1433-7479

文摘

In this paper, the robust decentralized stabilization of continuous-time uncertain nonlinear systems with multi control stations is developed using a neural network based online optimal control approach. The novelty lies in that the well-known adaptive dynamic programming method is extended to deal with the nonlinear feedback control problem under uncertain and large-scale environment. Through introducing an appropriate bounded function and defining a modified cost function, it can be observed that the decentralized optimal controller of the nominal system can achieve robust decentralized stabilization of original uncertain system. Then, a critic neural network is constructed for solving the modified Hamilton–Jacobi–Bellman equation corresponding to the nominal system in an online fashion. The weights of the critic network are tuned based on the standard steepest descent algorithm with an additional term provided to guarantee the boundedness of system states. The stability analysis of the closed-loop system is carried out via the Lyapunov approach. At last, two simulation examples are given to verify the effectiveness of the present control approach.