Discrete-Time Optimal Control via Local Policy Iteration Adaptive Dynamic Programming

Qinglai Wei, Derong Liu, Qiao Lin, Ruizhuo Song

Research output: Journal PublicationArticlepeer-review

92 Citations (Scopus)


In this paper, a discrete-time optimal control scheme is developed via a novel local policy iteration adaptive dynamic programming algorithm. In the discrete-time local policy iteration algorithm, the iterative value function and iterative control law can be updated in a subset of the state space, where the computational burden is relaxed compared with the traditional policy iteration algorithm. Convergence properties of the local policy iteration algorithm are presented to show that the iterative value function is monotonically nonincreasing and converges to the optimum under some mild conditions. The admissibility of the iterative control law is proven, which shows that the control system can be stabilized under any of the iterative control laws, even if the iterative control law is updated in a subset of the state space. Finally, two simulation examples are given to illustrate the performance of the developed method.

Original languageEnglish
Article number7515142
Pages (from-to)3367-3379
Number of pages13
JournalIEEE Transactions on Cybernetics
Issue number10
Publication statusPublished - Oct 2017
Externally publishedYes


  • Adaptive critic designs
  • adaptive dynamic programming (ADP)
  • approximate dynamic programming
  • local policy iteration
  • neuro-dynamic programming
  • nonlinear systems
  • optimal control

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Discrete-Time Optimal Control via Local Policy Iteration Adaptive Dynamic Programming'. Together they form a unique fingerprint.

Cite this