Reinforcement learning (RL) methods have demonstrated their efficiency in simulation. However, many of the applications for which RL offers great potential, such as autonomous driving, are also ...safety critical and require a certified closed-loop behavior in order to meet the safety specifications in the presence of physical constraints. This article introduces a concept called probabilistic model predictive safety certification (PMPSC), which can be combined with any RL algorithm and provides provable safety certificates in terms of state and input chance constraints for potentially large-scale systems. The certificate is realized through a stochastic tube that safely connects the current system state with a terminal set of states that is known to be safe. A novel formulation allows a recursively feasible real-time computation of such probabilistic tubes, despite the presence of possibly unbounded disturbances. A design procedure for PMPSC relying on Bayesian inference and recent advances in probabilistic set invariance is presented. Using a numerical car simulation, the method and its design procedure are illustrated by enhancing an RL algorithm with safety certificates.
This letter presents a stochastic model predictive control approach (MPC) for linear discrete-time systems subject to unbounded and correlated additive disturbance sequences, which makes use of the ...scenario approach for offline computation of probabilistic reachable sets. These sets are used in a tube-based MPC formulation, resulting in low computational requirements. Using a recently proposed MPC initialization scheme and nonlinear tube controllers, we provide recursive feasibility and closed-loop chance constraint satisfaction, as well as hard input constraint guarantees, which are typically challenging in tube-based formulations with unbounded noise. The approach is demonstrated in simulation for the control of an overhead crane system.
Gaussian process (GP) regression has been widely used in supervised machine learning due to its flexibility and inherent ability to describe uncertainty in function estimation. In the context of ...control, it is seeing increasing use for modeling of nonlinear dynamical systems from data, as it allows the direct assessment of residual model uncertainty. We present a model predictive control (MPC) approach that integrates a nominal system with an additive nonlinear part of the dynamics modeled as a GP. We describe a principled way of formulating the chance-constrained MPC problem, which takes into account residual uncertainties provided by the GP model to enable cautious control. Using additional approximations for efficient computation, we finally demonstrate the approach in a simulation example, as well as in a hardware implementation for autonomous racing of remote-controlled race cars with fast sampling times of 20 ms, highlighting improvements with regard to both performance and safety over a nominal controller.
We present a stochastic model predictive control (MPC) method for linear discrete-time systems subject to possibly unbounded and correlated additive stochastic disturbance sequences. Chance ...constraints are treated in analogy to robust MPC using the concept of probabilistic reachable sets for constraint tightening. We introduce an initialization of each MPC iteration which is always recursively feasible and guarantees chance constraint satisfaction for the closed-loop system, which is typically challenging for systems under unbounded disturbances. Under an i.i.d. zero-mean assumption, we provide an average asymptotic performance bound. A building control example illustrates the approach in an application with time-varying, correlated disturbances.
In this letter, we present a learning-based control approach for autonomous racing with an application to the AMZ Driverless race car gotthard . One major issue in autonomous racing is that accurate ...vehicle models that cover the entire performance envelope of a race car are highly nonlinear, complex, and complicated to identify, rendering them impractical for control. To address this issue, we employ a relatively simple nominal vehicle model, which is improved based on measurement data and tools from machine learning.The resulting formulation is an online learning data-driven model predictive controller, which uses Gaussian processes regression to take residual model uncertainty into account and achieve safe driving behavior. To improve the vehicle model online, we select from a constant in-flow of data points according to a criterion reflecting the information gain, and maintain a small dictionary of 300 data points. The framework is tested on the full-size AMZ Driverless race car, where it is able to improve the vehicle model and reduce lap times by <inline-formula><tex-math notation="LaTeX"> {\mathbf{10}{\%}}</tex-math></inline-formula> while maintaining safety of the vehicle.
High-precision trajectory tracking is fundamental in robotic manipulation. While industrial robots address this through stiffness and high-performance hardware, compliant and cost-effective robots ...require advanced control to achieve accurate position tracking. In this letter, we present a model-based control approach, which makes use of data gathered during operation to improve the model of the robotic arm and thereby the tracking performance. The proposed scheme is based on an inverse dynamics feedback linearization and a data-driven error model, which are integrated into a model predictive control formulation. In particular, we show how offset-free tracking can be achieved by augmenting a nominal model with both a Gaussian process, which makes use of offline data, and an additive disturbance model suitable for efficient online estimation of the residual disturbance via an extended Kalman filter. The performance of the proposed offset-free GPMPC scheme is demonstrated on a compliant 6 degrees of freedom robotic arm, showing significant performance improvements compared to other robot control algorithms.
Series elastic actuators decouple stiff motors and gear trains from the load by an mechanic elastic element. This elastic element can be used as torque sensor, acts as an energy storage, decouples ...the actuator from exogenous high frequency excitation and contributes towards shock resistance and safety in human–robot interaction scenarios. A series elastic element, however, fundamentally limits the achievable actuator bandwidth. Variable stiffness actuators (VSA) were introduced to overcome bandwidth limitations in series elastic actuators and to provide the flexibility necessary for energy efficient operation. The variable elastic element, however, complicates the design of torque and impedance controllers, which have to be synthesised by employing contradicting design objectives, such as minimisation of the output impedance, robust stability and performance. To overcome these synthesis problems, we present a new controller design procedure that imposes a positive real constraint on the load output port function to guarantee a stable interaction with respect to a passive, yet otherwise unknown environment. Additional design requirements are subsequently cast into a generalised plant. A H∞ design procedure is employed to design a gain-scheduled torque controller, which adapts to the varying mechanical stiffness of the actuator. It is then extended with a new procedure for the parametrisation of a superimposed impedance control-loop. The space of parameters for the impedance controller is determined in such way that it guarantees a positive real actuator output port function. The control strategy is tested in an in silico environment and in experiments on a test-bench with the Mechanical-Rotary Impedance Actuator (MeRIA).
Learning-based model predictive control has been widely applied in autonomous racing to improve the closed-loop behaviour of vehicles in a data-driven manner. When environmental conditions change, ...e.g., due to rain, often only the predictive model is adapted, but the controller parameters are kept constant. However, this can lead to suboptimal behaviour. In this paper, we address the problem of data-efficient controller tuning, adapting both the model and objective simultaneously. The key novelty of the proposed approach is that we leverage a learned dynamics model to encode the environmental condition as a so-called context. This insight allows us to employ contextual Bayesian optimization to efficiently transfer knowledge across different environmental conditions. Consequently, we require fewer data to find the optimal controller configuration for each context. The proposed framework is extensively evaluated with more than 3'000 laps driven on an experimental platform with 1:28 scale RC race cars. The results show that our approach successfully optimizes the lap time across different contexts requiring fewer data compared to other approaches based on standard Bayesian optimization.
This paper presents an adaptive high performance control method for autonomous miniature race cars. Racing dynamics are notoriously hard to model from first principles, which is addressed by means of ...a cautious nonlinear model predictive control (NMPC) approach that learns to improve its dynamics model from data and safely increases racing performance. The approach makes use of a Gaussian Process (GP) and takes residual model uncertainty into account through a chance constrained formulation. We present a sparse GP approximation with dynamically adjusting inducing inputs, enabling a real-time implementable controller. The formulation is demonstrated in simulations, which show significant improvement with respect to both lap time and constraint satisfaction compared to an NMPC without model learning.
PDF
