This note studies the adaptive optimal output regulation problem for continuous-time linear systems, which aims to achieve asymptotic tracking and disturbance rejection by minimizing some predefined ...costs. Reinforcement learning and adaptive dynamic programming techniques are employed to compute an approximated optimal controller using input/partial-state data despite unknown system dynamics and unmeasurable disturbance. Rigorous stability analysis shows that the proposed controller exponentially stabilizes the closed-loop system and the output of the plant asymptotically tracks the given reference signal. Simulation results on a LCL coupled inverter-based distributed generation system demonstrate the effectiveness of the proposed approach.
This paper proposes a novel data-driven control approach to address the problem of adaptive optimal tracking for a class of nonlinear systems taking the strict-feedback form. Adaptive dynamic ...programming (ADP) and nonlinear output regulation theories are integrated for the first time to compute an adaptive near-optimal tracker without any a priori knowledge of the system dynamics. Fundamentally different from adaptive optimal stabilization problems, the solution to a Hamilton-Jacobi-Bellman (HJB) equation, not necessarily a positive definite function, cannot be approximated through the existing iterative methods. This paper proposes a novel policy iteration technique for solving positive semidefinite HJB equations with rigorous convergence analysis. A two-phase data-driven learning method is developed and implemented online by ADP. The efficacy of the proposed adaptive optimal tracking control methodology is demonstrated via a Van der Pol oscillator with time-varying exogenous signals.
This article studies the distributed optimal output agreement problem for multiagent systems described by uncertain nonlinear models. By using the partial information of an objective function, the ...design aims to steer the outputs of the agents to an agreement on the optimal solution to the objective function. To solve this problem, this article introduces distributed coordinators to calculate the desired outputs, and designs reference-tracking controllers for the agents to follow the desired outputs. To deal with the nonlinear uncertain dynamics, the closed-loop multiagent system is considered as a dynamical network, and Sontag's input-to-state stability is employed to characterize the interconnections. It is shown that output agreement in multiagent nonlinear systems is achievable by means of distributed optimal controllers via a small-gain approach. The proposed design features a three-layer architecture, and the reference-tracking controllers can be implemented as successive nonlinear proportional-integral loops. A numerical example is employed to show the effectiveness of the design.
The event-based control strategy is an effective methodology for tackling the distributed control of multi-agent systems with limited on-board resources. This technical note focuses on event-based ...leader-following consensus for multi-agent systems described by general linear models and subject to input time delay between controller and actuator. For each agent, the controller updates are event-based and only triggered at its own event times. A necessary condition and two sufficient conditions on leader-following consensus are presented, respectively. It is shown that continuous communication between neighboring agents can be avoided and the Zeno-behavior of triggering time sequences is excluded. A numerical example is presented to illustrate the effectiveness of the obtained theoretical results.
This paper presents a new approach to event-triggered control for nonlinear uncertain systems by using the notion of input-to-state stability (ISS) and the nonlinear small-gain theorem. The ...contribution of this paper is threefold. First, it is proved that infinitely fast sampling can be avoided if the system is input-to-state stabilizable with the sampling error as the external input and the corresponding ISS gain is locally Lipschitz. No assumption on the existence of known ISS-Lyapunov functions is made in the discussions. Moreover, the forward completeness problem with event-triggered control is studied systematically by using ISS small-gain arguments. Second, the proposed approach gives rise to a new self-triggered sampling strategy for a class of nonlinear systems subject to external disturbances. If an upper bound of the external disturbance is known, then the closed-loop system can be designed to be robust to the external disturbance, and moreover, the system state globally asymptotically converges to the origin if the external disturbance decays to zero. Third, a new design method is developed for event-triggered control of nonlinear uncertain systems in the strict-feedback form. It is particularly shown that the ISS gain with the sampling error as the input can be designed to satisfy the proposed condition for event-triggered control and self-triggered control.
In this paper, a data-driven non-model-based approach is proposed for the adaptive optimal control of a class of connected vehicles that is composed of n human-driven vehicles only transmitting ...motional data and an autonomous vehicle in the tail receiving the broadcasted data from preceding vehicles by wireless vehicle-to-vehicle (V2V) communication devices. Considering the cases of range-limited V2V communication and input saturation, several optimal control problems are formulated to minimize the errors of distance and velocity and to optimize the fuel usage. By employing an adaptive dynamic programming technique, the optimal controllers are obtained without relying on the knowledge of system dynamics. The effectiveness of the proposed approaches is demonstrated via the online learning control of the connected vehicles in Paramics' traffic microsimulation.
This paper presents a novel method of global adaptive dynamic programming (ADP) for the adaptive optimal control of nonlinear polynomial systems. The strategy consists of relaxing the problem of ...solving the Hamilton-Jacobi-Bellman (HJB) equation to an optimization problem, which is solved via a new policy iteration method. The proposed method distinguishes from previously known nonlinear ADP methods in that the neural network approximation is avoided, giving rise to significant computational improvement. Instead of semiglobally or locally stabilizing, the resultant control policy is globally stabilizing for a general class of nonlinear polynomial systems. Furthermore, in the absence of the a priori knowledge of the system dynamics, an online learning method is devised to implement the proposed policy iteration technique by generalizing the current ADP theory. Finally, three numerical examples are provided to validate the effectiveness of the proposed method.
This paper studies event-based control of nonlinear systems with state quantization. Two configurations of the event-based quantized controller are considered: Quantization after sampling and ...quantization before sampling. The considered quantizer is assumed to have a finite quantization range. With input-to-state stability (ISS) tools, new event-triggering, and dynamic quantization mechanisms are designed to deal with the interaction of the quantizer and the sampler. For both of the configurations, infinitely fast sampling, and in particular the Zeno phenomenon, is avoided. And the system state asymptotically converges to the origin, if a growth condition is satisfied by the ISS gain of the controlled system.
This paper presents a novel non-model-based, data-driven adaptive optimal controller design for linear continuous-time systems with completely unknown dynamics. Inspired by the stochastic ...approximation theory, a continuous-time version of the traditional value iteration (VI) algorithm is presented with rigorous convergence analysis. This VI method is crucial for developing new adaptive dynamic programming methods to solve the adaptive optimal control problem and the stochastic robust optimal control problem for linear continuous-time systems. Fundamentally different from existing results, the a priori knowledge of an initial admissible control policy is no longer required. The efficacy of the proposed methodology is illustrated by two examples and a brief comparative study between VI and earlier policy-iteration methods.
This article studies the robustness of policy iteration in the context of continuous-time infinite-horizon linear quadratic regulator (LQR) problem. It is shown that Kleinman's policy iteration ...algorithm is small-disturbance input-to-state stable, a property that is stronger than Sontag's local input-to-state stability but weaker than global input-to-state stability. More precisely, whenever the error in each iteration is bounded and small, the solutions of the policy iteration algorithm are also bounded and enter a small neighborhood of the optimal solution of the LQR problem. Based on this result, an off-policy data-driven policy iteration algorithm for the LQR problem is shown to be robust when the system dynamics are subject to small additive unknown bounded disturbances. The theoretical results are validated by a numerical example.
PDF
