In this paper, a new iterative adaptive dynamic programming (ADP) method is proposed to solve a class of continuous-time nonlinear two-person zero-sum differential games. The idea is to use the ADP ...technique to obtain the optimal control pair iteratively which makes the performance index function reach the saddle point of the zero-sum differential games. If the saddle point does not exist, the mixed optimal control pair is obtained to make the performance index function reach the mixed optimum. Stability analysis of the nonlinear systems is presented and the convergence property of the performance index function is also proved. Two simulation examples are given to illustrate the performance of the proposed method.
This article is concerned with a new generalized actor-critic learning (GACL) optimal control method. It aims at the optimal energy control and management for smart home systems, which is expected to ...minimize the consumption cost for home users. In the present GACL optimal control method, it is the first time that three iteration processes, which are global iteration, local iteration, and interior iteration, respectively, are established to obtain the optimal energy control law. The main contribution of the developed method is to establish a common iteration structure for both value and policy iterations in adaptive dynamic programming based on a control law sequence in each iteration for periodic time-varying systems, instead of a single control law, and simultaneously accelerates the convergence rate. The monotonicity, convergence, and optimality of the iterative value function for the GACL optimal control method are proven. Finally, numerical results and comparisons are displayed to show the superiority of the developed method.
In this paper, an event-triggered near optimal control structure is developed for nonlinear continuous-time systems with control constraints. Due to the saturating actuators, a nonquadratic cost ...function is introduced and the Hamilton-Jacobi-Bellman (HJB) equation for constrained nonlinear continuous-time systems is formulated. In order to solve the HJB equation, an actor-critic framework is presented. The critic network is used to approximate the cost function and the action network is used to estimate the optimal control law. In addition, in the proposed method, the control signal is transmitted in an aperiodic manner to reduce the computational and the transmission cost. Both the networks are only updated at the trigger instants decided by the event-triggered condition. Detailed Lyapunov analysis is provided to guarantee that the closed-loop event-triggered system is ultimately bounded. Three case studies are used to demonstrate the effectiveness of the proposed method.
An event-triggered output feedback control approach is proposed via a disturbance observer and adaptive dynamic programming (ADP). The solution starts from constructing a nonlinear disturbance ...observer which only depends on the measurement of system output. A state observer is then developed based on approximation information of system dynamics via neural networks. In order to avoid continuous transmission and reduce communication burden in the closed-loop system, an event-triggered mechanism is introduced such that the control signal is updated only at specific instant that a triggered condition is violated. By virtue of the disturbance observer and state observer, an output-feedback ADP control approach then is developed, where only a critic network is employed to estimate the value function. Based on Lyapunov stability theory, stability of the closed-loop system is rigorously analyzed, and the effectiveness of the proposed control approach is verified by two simulation examples.
As a powerful method of solving the nonlinear optimal control problem, the iterative adaptive dynamic programming (IADP) is usually established on the known controlled system model and is particular ...for affine nonlinear systems. Since most nonlinear systems are complicated to establish accurate mathematical models, this paper provides a novel data-based approximate optimal control algorithm, named iterative neural dynamic programming (INDP) for affine and non-affine nonlinear systems by using system data rather than accurate system models. The INDP strategy is built within the framework of IADP, where the convergence guarantee of the iteration is provided. The INDP algorithm is implemented based on the model-based heuristic dynamic programming (HDP) structure, where model, action and critic neural networks are employed to approximate the system dynamics, the control law and the iterative cost function, respectively. During the back-propagation of action and critic networks, the approach of directly minimizing the iterative cost function is developed to eliminate the requirement of establishing system models. The neural network implementation of the INDP algorithm is presented in detail and the associated stability is also analyzed. Simulation studies are conducted on affine and non-affine nonlinear systems, and further on the manipulator system, where all results have demonstrated the effectiveness of the proposed data-based approximate optimal control method.
This paper focuses on the problem of optimal fuzzy output-feedback tracking control for dynamic positioning of marine vessels simultaneously with model uncertainties, unknown disturbances, ...unavailable velocities and thruster saturations. The control scheme is built by integrating a fuzzy velocity observer, a finite-time disturbance observer, a dynamic auxiliary system, the optimal control strategy with the dynamic surface control. The fuzzy velocity observer is constructed to obtain the unavailable velocities without requiring the certain model dynamics. The finite-time disturbance observer is established to estimate the unknown disturbances. The dynamic auxiliary system is designed to compensate for thruster saturations effects. By incorporating these methods, a disturbance rejection adaptive controller is designed with dynamic surface control. The optimal compensation term is inserted to minimize the cost function of the tracking error system by adaptive dynamic programming. It is shown that the optimal control consisting of the adaptive controller and the optimal compensation term guarantees that all signals in the closed-loop dynamic positioning system are bounded. Finally, simulation results demonstrate the effectiveness of the proposed optimal control scheme.
This note studies the adaptive optimal output regulation problem for continuous-time linear systems, which aims to achieve asymptotic tracking and disturbance rejection by minimizing some predefined ...costs. Reinforcement learning and adaptive dynamic programming techniques are employed to compute an approximated optimal controller using input/partial-state data despite unknown system dynamics and unmeasurable disturbance. Rigorous stability analysis shows that the proposed controller exponentially stabilizes the closed-loop system and the output of the plant asymptotically tracks the given reference signal. Simulation results on a LCL coupled inverter-based distributed generation system demonstrate the effectiveness of the proposed approach.
This paper proposes an approximate dynamic programming (ADP)-based approach for the economic dispatch (ED) of microgrid with distributed generations. The time-variant renewable generation, ...electricity price, and the power demand are considered as stochastic variables in this paper. An ADP based ED (ADPED) algorithm is proposed to optimally operate the microgrid under these uncertainties. To deal with the uncertainties, Monte Carlo method is adopted to sample the training scenarios to give empirical knowledge to ADPED. The piecewise linear function (PLF) approximation with improved slope updating strategy is employed for the proposed method. With sufficient information extracted from these scenarios and embedded in the PLF function, the proposed ADPED algorithm can not only be used in day-ahead scheduling but also the intra-day optimization process. The algorithm can make full use of historical prediction error distribution to reduce the influence of inaccurate forecast on the system operation. Numerical simulations demonstrate the effectiveness of the proposed approach. The near-optimal decision obtained by ADPED is very close to the global optimality. And it can be adaptive to both day-ahead and intra-day operation under uncertainty.
This article considers the semiglobal cooperative suboptimal output regulation problem of heterogeneous multi-agent systems with unknown agent dynamics in the presence of input saturation. To solve ...the problem, we develop distributed suboptimal control strategies from two perspectives, namely, model-based and data-driven. For the model-based case, we design a suboptimal control strategy by using the low-gain technique and output regulation theory. Moreover, when the agents' dynamics are unknown, we design a data-driven algorithm to solve the problem. We show that proposed control strategies ensure each agent's output gradually follows the reference signal and achieves interference suppression while guaranteeing closed-loop stability. The theoretical results are illustrated by a numerical simulation example.
PDF
