Recent successes in the field of machine learning, as well as the availability of increased sensing and computational capabilities in modern control systems, have led to a growing interest in ...learning and data-driven control techniques. Model predictive control (MPC), as the prime methodology for constrained control, offers a significant opportunity to exploit the abundance of data in a reliable manner, particularly while taking safety constraints into account. This review aims at summarizing and categorizing previous research on learning-based MPC, i.e., the integration or combination of MPC with learning methods, for which we consider three main categories. Most of the research addresses learning for automatic improvement of the prediction model from recorded data. There is, however, also an increasing interest in techniques to infer the parameterization of the MPC controller, i.e., the cost and constraints, that lead to the best closed-loop performance. Finally, we discuss concepts that leverage MPC to augment learning-based controllers with constraint satisfaction properties.
Normal pressure hydrocephalus (NPH) is a chronic and progressive disease that affects predominantly elderly subjects. The most prevalent symptoms are gait disorders, generally determined by visual ...observation or measurements taken in complex laboratory environments. However, controlled testing environments can have a significant influence on the way subjects walk and hinder the identification of natural walking characteristics. The study aimed to investigate the differences in walking patterns between a controlled environment (10 m walking test) and real-world environment (72 h recording) based on measurements taken via a wearable gait assessment device. We tested whether real-world environment measurements can be beneficial for the identification of gait disorders by performing a comparison of patients’ gait parameters with an aged-matched control group in both environments. Subsequently, we implemented four machine learning classifiers to inspect the individual strides’ profiles. Our results on twenty young subjects, twenty elderly subjects and twelve NPH patients indicate that patients exhibited a considerable difference between the two environments, in particular gait speed (p-value p=0.0073), stride length (p-value p=0.0073), foot clearance (p-value p=0.0117) and swing/stance ratio (p-value p=0.0098). Importantly, measurements taken in real-world environments yield a better discrimination of NPH patients compared to the controlled setting. Finally, the use of stride classifiers provides promise in the identification of strides affected by motion disorders.
This brief presents an inverse optimal control methodology and its application to training a predictive model of human motor control from a manipulation task. It introduces a convex formulation for ...learning both objective function and constraints of an infinite-horizon constrained optimal control problem with nonlinear system dynamics. The inverse approach utilizes Bellman's principle of optimality to formulate the infinite-horizon optimal control problem as a shortest path problem and the Lagrange multipliers to identify constraints. We highlight the key benefit of using the shortest path formulation, i.e., the possibility of training the predictive model with short and selected trajectory segments. The method is applied to training a predictive model of movements of a human subject from a manipulation task. The study indicates that individual human movements can be predicted with low error using an infinite-horizon optimal control problem with constraints on the shoulder movement.
This paper addresses the trajectory-tracking problem under uncertain road-surface conditions for autonomous vehicles. We propose a stochastic nonlinear model predictive controller (SNMPC) that learns ...a tyre-road friction model online using standard automotive-grade sensors. Learning the entire tyre-road friction model in real time requires driving in the nonlinear, potentially unstable regime of the vehicle dynamics, using a prediction model that may not have fully converged. To handle this, we formulate the tyre-friction model learning in a Bayesian framework and propose two estimators that learn different aspects of the tyre-road friction. The estimators output the estimate of the tyre-friction model as well as the uncertainty of the estimate, which expresses the confidence in the model for different driving regimes. The SNMPC exploits the uncertainty estimate in its prediction model to take proper action when the uncertainty is large. We validate the approach in an extensive Monte Carlo study using real vehicle parameters and in CarSim. The results when comparing to various MPC approaches indicate a substantial reduction in constraint violations, as well as a reduction in closed-loop cost. We also demonstrate the real-time feasibility in automotive-grade processors using a dSPACE MicroAutoBox-II rapid prototyping unit, showing a worst-case computation time of roughly 40 ms.
This article proposes a method for calibrating control parameters. The examples of such control parameters are gains of proportional-integral-derivative (PID) controllers, weights of a cost function ...for optimal control, filter coefficients, the sliding surface of a sliding mode controller, or weights of a neural network. Hence, the proposed method can be applied to a wide range of controllers. The method uses a Kalman filter that estimates control parameters, using data of closed-loop system operation. The control parameter calibration is driven by a training objective, which encompasses specifications on the performance of the dynamical system. The performance-driven calibration method tunes the parameters online and robustly, is computationally efficient, has low data storage requirements, and is easy to implement, making it appealing for many real-time applications. Simulation results show that the method is able to learn control parameters quickly, is able to tune the parameters to compensate for disturbances, and is robust to noise. A simulation study with the high-fidelity vehicle simulator CarSim shows that the method can calibrate controllers of a complex dynamical system online, which indicates its applicability to a real-world system. We also verify the real-time feasibility on an embedded platform with automotive-grade processors by implementing our method on a dSPACE MicroAutoBox-II rapid prototyping unit.
This article presents a method for tailoring a parametric controller based on human ratings. The method leverages supervised learning concepts in order to train a reward model from data. It is ...applied to a gait rehabilitation robot with the goal of teaching the robot how to walk patients physiologically. In this context, the reward model judges the physiology of the gait cycle (instead of therapists) using sensor measurements provided by the robot and the automatic feedback controller chooses the input settings of the robot to maximize the reward. The key advantage of the proposed method is that only a few input adaptations are necessary to achieve a physiological gait cycle. Experiments with nondisabled subjects show that the proposed method permits the incorporation of human expertise into a control law and to automatically walk patients physiologically.
•An analytical approach to evaluate the volume integrals emerging in Lagrangian–Eulerian methods is proposed.•The proposed strategy allows to evaluate particles modeled as arbitrary convex polyhedra ...with polynomial filtering functions.•A generic strategy and simplifications are proposed to accommodate both structured and unstructured grids.
This paper presents an analytical approach to evaluate the volume integrals emerging during dispersed phase fraction computation in Lagrangian–Eulerian methods. It studies a zeroth, second, and fourth order polynomial filtering function in test cases featuring structured and unstructured grids. The analytical integration is enabled in three steps. First, the divergence theorem is applied to transform the volume integral into surface integrals over the volumes’ boundaries. Secondly, the surfaces are projected alongside the first divergence direction. Lastly, the divergence theorem is applied for the second time to transform the surface integrals into line integrals. We propose a generic strategy and simplifications to derive an analytical description of the complex geometrical entities such as non-planar surfaces. This strategy enables a closed solution to the line integrals for polynomial filtering functions. Furthermore, this paper shows that the proposed approach is suitable to handle unstructured grids. A sine wave and Gaussian filtering function is tested and the fourth order polynomial is found to be a good surrogate for the sine wave filtering function as no expensive trigonometric evaluations are necessary.
We consider the problem of predicting the motion of vehicles in the surrounding of an autonomous car, for improved motion planning in lane-based driving scenarios without inter-vehicle communication. ...First, we address the problem of single-vehicle estimation by designing a filtering scheme based on an Interacting Multiple Model Kalman Filter equipped with novel intention-based models. Second, we augment the proposed scheme with an optimization-based projection that enables the generation of non-colliding predictions. We then extend the approach to the problem of simultaneously estimating multiple vehicles by using a hierarchical approach based on a priority list. The priority list is dynamically adapted in real-time according to a proposed sorting algorithm. Finally, we evaluate the proposed scheme in simulations using real-life vehicle data from the Next Generation Simulation (NGSIM) dataset.
Powered lower limb exoskeletons are a viable solution for people with a spinal cord injury to regain mobility for their daily activities. However, the commonly employed rigid actuation and ...pre-programmed trajectories increase the risk of falling in case of collisions with external objects. Compliant actuation may reduce forces during collisions, thus protecting hardware and user. However, experimental data of collisions specific to lower limb exoskeletons are not available. In this work, we investigated how a variable stiffness actuator at the knee joint influences collision forces transmitted to the user via the exoskeleton. In a test bench experiment, we compared three configurations of an exoskeleton leg with a variable stiffness knee actuator in (i) compliant or (ii) stiff configurations, and with (iii) a rigid actuator. The peak torque observed at the pelvis was reduced from 260.2 Nm to 116.2 Nm as stiffness decreased. In addition, the mechanical impulse was reduced by a factor of three. These results indicate that compliance in the knee joint of an exoskeleton can be favorable in case of collision and should be considered when designing powered lower limb exoskeletons. Overall, this could decrease the effort necessary to maintain balance after a collision, and improved collision handling in exoskeletons could result in safer use and benefit their usefulness in daily life.
This letter presents an approach for auto-tuning feedback controllers and online trajectory planners to achieve robust locomotion of a legged robot. The auto-tuning approach uses an Unscented Kalman ...Filter (UKF) formulation, which adapts/calibrates control parameters online using a recursive implementation. In particular, this letter shows how to use the auto-tuning approach to calibrate cost function weights of a Model Predictive Control (MPC) stance controller and feedback gains of a swing controller for a quadruped robot. Furthermore, this letter extends the auto-tuning approach to calibrating parameters of an online trajectory planner, where the height of a swing leg and the robot's walking speed are optimized, while minimizing its energy consumption and foot slippage. This allows us to generate stable reference trajectories online and in real time. Results using a high-fidelity Unitree A1 robot simulator in Gazebo provided by the robot manufacturer show the advantages of using auto-tuning for calibrating feedback controllers and for computing reference trajectories online for reduced development time and improved tracking performance.
PDF
