Adaptive Dynamic Programming for Discrete-Time Zero-Sum Games

Qinglai Wei, Derong Liu, Qiao Lin, Ruizhuo Song

Research output: Journal PublicationArticlepeer-review

126 Citations (Scopus)


In this paper, a novel adaptive dynamic programming (ADP) algorithm, called “iterative zero-sum ADP algorithm,” is developed to solve infinite-horizon discrete-time two-player zero-sum games of nonlinear systems. The present iterative zero-sum ADP algorithm permits arbitrary positive semidefinite functions to initialize the upper and lower iterations. A novel convergence analysis is developed to guarantee the upper and lower iterative value functions to converge to the upper and lower optimums, respectively. When the saddle-point equilibrium exists, it is emphasized that both the upper and lower iterative value functions are proved to converge to the optimal solution of the zero-sum game, where the existence criteria of the saddle-point equilibrium are not required. If the saddle-point equilibrium does not exist, the upper and lower optimal performance index functions are obtained, respectively, where the upper and lower performance index functions are proved to be not equivalent. Finally, simulation results and comparisons are shown to illustrate the performance of the present method.
Original languageEnglish
Pages (from-to)957-969
JournalIEEE Transactions on Neural Networks and Learning Systems
Issue number4
Publication statusPublished - 1 Apr 2018


  • Adaptive critic designs
  • adaptive dynamic programming (ADP)
  • approximate dynamic programming
  • neurodynamic programming
  • optimal control
  • zero-sum game


Dive into the research topics of 'Adaptive Dynamic Programming for Discrete-Time Zero-Sum Games'. Together they form a unique fingerprint.

Cite this