This research proposes a reinforcement learning-based algorithm and a deep reinforcement learning-based algorithm for energy management of a series hybrid electric tracked vehicle. Firstly, the ...powertrain model of the series hybrid electric tracked vehicle (SHETV) is constructed, then the corresponding energy management formulation is established. Subsequently, a new variant of reinforcement learning (RL) method Dyna, namely Dyna-H, is developed by combining the heuristic planning step with the Dyna agent and is applied to energy management control for SHETV. Its rapidity and optimality are validated by comparing with DP and conventional Dyna method. Facing the problem of the “curse of dimensionality” in the reinforcement learning method, a novel deep reinforcement learning algorithm deep Q-learning (DQL) is designed for energy management control, which uses a new optimization method (AMSGrad) to update the weights of the neural network. Then the proposed deep reinforcement learning control system is trained and verified by the realistic driving condition with high-precision, and is compared with the benchmark method DP and the traditional DQL method. Results show that the proposed deep reinforcement learning method realizes faster training speed and lower fuel consumption than traditional DQL policy does, and its fuel economy quite approximates to global optimum. Furthermore, the adaptability of the proposed method is confirmed in another driving schedule.
•The powertrain model of the series hybrid electric tracked vehicle is constructed.•A novel reinforcement learning-based energy management strategy is proposed.•The rapidity and optimality of the reinforcement learning method are validated.•A new optimization method is applied to update the weights of the neural network.•The proposed deep reinforcement learning method realizes better performance.
In this paper, a complete mathematical model for a half-tracked vehicle with eight degrees of freedom is established. It is difficult to control a suspension system owing to complexity of system ...itself and irregular excitations from road surfaces. A road excitation model is analyzed in which a relationship between the vehicle body and the gun is taken into account. Active disturbance rejection controllers are proposed in all channels of six independent subsystems by decoupling the original system. With switch extended state observers, total disturbances are eliminated for the half-tracked vehicle. Simulation results are presented to illustrate effectiveness of the proposed method in this paper.
In this paper, a bi-level control framework is proposed to improve the energy efficiency for a hybrid tracked vehicle. The higher-level discusses how to accurately predict power demand based on the ...Markov Chain. Specially, fuzzy encoding predictor is used for power demand prediction, and a real-time recursive algorithm is applied to fuse the future power demand information into transition probability matrix (TPM) computation. Furthermore, the Kullback-Leibler (KL) divergence rate is employed to decide the alteration of control strategy. The lower-level computes the relevant energy management strategy, based on the updated TPM and a model-free reinforcement learning (RL) technique. Simulation results illustrate that the vehicular energy efficiency in the proposed scheme exceeds the common RL control by tuning the KL divergence value. Comparative results also show that the developed control strategy outperforms the common RL one, in terms of energy efficiency and computational speed.
•KAMPC takes the kinematics and trajectory tracking control as a combined problem.•KAMPC is the exploration process of intrinsic characteristics for autonomous systems.•We verified the algorithm from ...both theoretical analysis and real experiment.•KAMPC can reduce the average tracking residuals and has good statistical performance.
Although accurate trajectory tracking of autonomous vehicles strongly depends on a precise model, it is difficult to establish a precise model for high-speed tracked vehicles under off-road conditions, due to the complicated interactions between the tracks and the terrain. This paper presents a novel trajectory tracking methodology, called kinematics-aware model predictive control (KAMPC), by combining the slip kinematic model with a trajectory tracking control strategy for skid-steered tracked vehicles. For the slip kinematic model, an online identification methodology called six-parameter slip parameter estimation (SSPE) algorithm is proposed based on the instantaneous centers of rotation. For the trajectory tracking control strategy, optimized algorithm model predictive control is used to obtain the optimal control inputs. Finally, our method is validated through extensive experiments based on a distributed electric-drive high-speed tracked vehicle test platform over different types of terrain and varied off-road surface conditions. Experiment results show that the tracked vehicle kinematic characteristics are distinctive in off-road conditions, and the slipping and skidding that occur when turning are critical to this work. Compared with the other three widely used types of methodologies, the present KAMPC approach can reduce the average tracking residuals and has the best statistical performance.
•A double deep Q-learning (DDQL) based energy management of HETV is proposed.•Lateral dynamics is taken into the decision system of reinforcement learning.•Simulation results show that the fuel ...economy of DDQL algorithm is 7.1% better than DQL.
An energy management strategy, based on double deep Q-learning algorithm, is proposed for a dual-motor driven hybrid electric tracked-vehicle. Typical model framework of tracked-vehicle is established where the lateral dynamic can be taken into consideration. For the propose of optimizing the fuel consumption performance, a double deep Q-learning-based control structure is put forward. Compared to conventional deep Q-learning, the proposed strategy prevents training process falling into the overoptimistic estimate of policy value and highlights its significant advantages in terms of the iterative convergence rate and optimization performance. Unique observation states are selected as input variables of reinforcement learning algorithm in view of revealing tracked-vehicles characteristic. The conventional deep Q-learning and dynamic programming are also employed and compared with the proposed strategy for different driving schedules. Simulation results demonstrate the fuel economy of proposed methodology achieves 7.1% better than that of conventional deep Q learning-based strategy and reaches 93.2% level of Dynamic programing benchmark. Moreover, the designed algorithm has a good performance in battery SOC retention with different initial values.
As an important component of land transportation systems, tracked vehicles (TRVs) and wheeled vehicles (WVs) have developed independently in parallel, particularly in the modeling of vehicle-ground ...interactions. However, their differences are not as significant as they appear. This paper introduces a simplified terramechanics-based track-ground interaction model for the motion control of TRVs on firm ground. The simplified interaction model addresses the problem that the terramechanicsbased models are too complex to be applied to optimizationbased real-time control algorithms. Interestingly, the proposed track-ground interaction model closely resembles to the tire model used for WVs. Through comparison, we present the unified mechanisms underlying vehicle-ground interactions. In our approach, TRVs can be treated as a special type of skid-steer WV, which benefits the theories and methods of wheel vehicles to be deployed in the TRV domain. Finally, we verify the proposed interaction model with extensive dynamic data from a real dual motor-driven TRV to demonstrate its effectiveness.
•During an anti-tank mine explosion, torsion bars are a threat to the crew.•Torsional Springs radically increase workspace inside the tracked vehicle.•Torsional Springs decrease mass of the tracked ...vehicle.•Hyperbolic torsion springs have the stiffness characteristics as the torsion bars.•This kind of torsional springs may radically change tracked vehicles construction.
In the article, the authors present an advanced model of hyperbolic torsion spring based on the Finite Element Method (FEM). This model takes into account the contact forces between the spring arms, which allows you to simulate its operation even in the case of a large twisting angle. On the basis of this FEM model, the spring stiffness characteristics were constructed. Based on this characteristics, the existing 2S1 track platform model has been rebuilt. In this model, a much higher mass of the body is allowed than in the original model. On the basis of the elastic spring model, parameters such as the static deflection of the suspension have also been determined, as well as its range of mobility compared to the original suspension based on torsion bars. In addition, spring deflection phases have been presented in terms of internal stresses, which are important for the strength analysis of the spring. The originality of the described solution lies in the unique design of the so-called a hyperbolic spring. This design has not been previously described in the literature.
•A systematic control-oriented model for the HTV was built.•A Markov chain model learns power transition probability recursively.•The Kullback–Leibler divergence rate determines the transition ...probability update.•Reinforcement learning (RL) was applied to optimize the control strategy.•The strategy improves fuel efficiency and works real time.
To realize the optimal energy allocation between the engine-generator and battery of a hybrid tracked vehicle (HTV), a reinforcement learning-based real-time energy-management strategy was proposed. A systematic control-oriented model for the HTV was built and validated through the test bench, including the battery pack, the engine-generator set (EGS), and the power request. To use effectively the statistical information of power request online, a Markov chain-based real-time power request recursive algorithm for learning transition probabilities was derived and validated. The Kullback–Leibler (KL) divergence rate was adopted to determine when the transition probability matrix and the optimal control strategy update in real time. Reinforcement learning (RL) was applied to compare quantitatively the effects of different forgetting factors and KL divergence rates on reducing fuel consumption. RL has also been used to optimize the control strategy for HTV, compared to preliminary and dynamic programming-based control strategies. The real-time and robust performance of the proposed online energy management strategy was verified under two driving schedules collected in the field test. The simulation results indicate the proposed RL-based energy management strategy can significantly improve fuel efficiency and can be applied in real time.
A reinforcement learning-based adaptive energy management (RLAEM) is proposed for a hybrid electric tracked vehicle (HETV) in this paper. A control oriented model of the HETV is first established, in ...which the state-of-charge (SOC) of battery and the speed of generator are the state variables, and the engine's torque is the control variable. Subsequently, a transition probability matrix is learned from a specific driving schedule of the HETV. The proposed RLAEM decides appropriate power split between the battery and engine-generator set (EGS) to minimize the fuel consumption over different driving schedules. With the RLAEM, not only is driver's power requirement guaranteed, but also the fuel economy is improved as well. Finally, the RLAEM is compared with the stochastic dynamic programming (SDP)-based energy management for different driving schedules. The simulation results demonstrate the adaptability, optimality, and learning ability of the RLAEM and its capacity of reducing the computation time.
Display omitted
•The overall model for the hybrid electric tracked vehicle is built in detail.•Fast Q-learning algorithm is applied to derive energy management strategy.•An efficient online energy ...management strategy update framework is constructed.•Hardware-in-loop simulation experiment is conducted to validate the performance.•The strategy improves fuel economy and has potential for real-time applications.
The energy management approach of hybrid electric vehicles has the potential to overcome the increasing energy crisis and environmental pollution by reducing the fuel consumption. This paper proposes an online updating energy management strategy to improve the fuel economy of hybrid electric tracked vehicles. As the basis of the research, the overall model for the hybrid electric tracked vehicle is built in detail and validated through the field experiment. To accelerate the convergence rate of the control policy calculation, a novel reinforcement learning algorithm called fast Q-learning is applied which improves the computational speed by 16%. The cloud-computation is presented to afford the main computation burden to realize the online updating energy management strategy in hardware-in-loop simulation bench. The Kullback-Leibler divergence rate to trigger the update of the control strategy is designed and realized in hardware-in-loop simulation bench. The simulation results show that the fuel consumption of the fast Q-learning based online updating strategy is 4.6% lower than that of stationary strategy, and is close to that of dynamic programming strategy. Besides, the computation time of the proposed method is only 1.35 s which is much shorter than that of dynamic programming based method. The results indicate that the proposed energy management strategy can greatly improve the fuel economy and have the potential to be applied in the real-time application. Moreover, the adaptability of the online energy management strategy is validated in three realistic driving schedules.
PDF
