For discrete-time nonlinear systems, this research is concerned with optimal control problems (OCPs) with constrained cost, and a novel value iteration with constrained cost (VICC) method is ...developed to solve the optimal control law with the constrained cost functions. The VICC method is initialized through a value function constructed by a feasible control law. It is proven that the iterative value function is nonincreasing and converges to the solution of the Bellman equation with constrained cost. The feasibility of the iterative control law is proven. The method to find the initial feasible control law is given. Implementation using neural networks (NNs) is introduced, and the convergence is proven by considering the approximation error. Finally, the property of the present VICC method is shown by two simulation examples.
The residential energy scheduling of solar energy is an important research area of smart grid. On the demand side, factors such as household loads, storage batteries, the outside public utility grid ...and renewable energy resources, are combined together as a nonlinear, time-varying, indefinite and complex system, which is difficult to manage or optimize. Many nations have already applied the residential real-time pricing to balance the burden on their grid. In order to enhance electricity efficiency of the residential micro grid, this paper presents an action dependent heuristic dynamic programming ( ADHDP ) method to solve the residential energy scheduling problem. The highlights of this paper are listed below. First, the weather-type classification is adopted to establish three types of programming models based on the features of the solar energy. In addition, the priorities of different energy resources are set to reduce the loss of electrical energy transmissions. Second, three ADHDP-based neural networks, which can update themselves during applications, are designed to manage the flows of electricity. Third, simulation results show that the proposed scheduling method has effectively reduced the total electricity cost and improved load balancing process. The comparison with the particle swarm optimization algorithm further proves that the present method has a promising effect on energy management to save cost.
•A general charging scheduling problem for an electric bus fleet, which can be used a building block of more complex electric bus system planning or operations problems.•Time-based charging windows, ...battery state-of-charge bounds, time-of-use charging tariffs, and station-specific electricity load capacities considered.•A mixed linear integer programming model with charging location and time decisions specified by location-specific charging time slots.•A Lagrangian relaxation framework for decomposing the fleet charging scheduling problem by individual buses.•A bi-criterion dynamic programming algorithm for each single-bus charging scheduling subproblem.
Electrification poses a promising low-carbon or even zero-carbon transportation solution, serving as a strategic approach to reducing carbon emissions and promoting carbon neutrality in the transportation sector. Along the transportation electrification pathway, the goal of carbon neutrality can be further accelerated with an increasing amount of electricity being generated from renewable energies. The past decade observed the rapid development of battery technologies and deployment of electricity infrastructure worldwide, fostering transportation electrification to expand from railways to light and then heavy vehicles on roadways. In China, a massive number of electric buses have been employed and operated in dozens of metropolises. An important daily operations issue with these urban electric buses is how to coordinate their charging activities in a cost-effective manner, considering various physical, financial, institutional, and managerial constraints.
This paper addresses a general charging scheduling problem for an electric bus fleet operated across multiple bus lines and charging depots and terminals, aiming at finding an optimal set of charging location and time decisions given the available charging windows. The charging windows for each bus are predetermined in terms of its layovers at depots and terminals and each of them is discretized into a number of charging slots with the same time duration. A mixed linear integer programming model with binary charging slot choice and continuous state-of-charge (SOC) variables is constructed for minimizing the total charging cost of the bus fleet subject to individual electricity consumption rates, electricity charging rates, time-based charging windows, battery SOC bounds, time-of-use (TOU) charging tariffs, and station-specific electricity load capacities. A Lagrangian relaxation framework is employed to decouple the joint charging schedule of a bus fleet into a number of independent single-bus charging schedules, which can be efficiently addressed by a bi-criterion dynamic programming algorithm. A real-world regional electric bus fleet of 122 buses in Shanghai, China is selected for validating the effectiveness and practicability of the proposed charging scheduling model and algorithm. The optimization results numerically reveal the impacts of TOU tariffs, station load capacities, charging infrastructure configurations, and battery capacities on the bus system performance as well as individual recharging behaviors, and justify the superior solution efficiency of our algorithm against a state-of-the-art commercial solver.
We consider a broad class of stochastic dynamic programming problems that are amenable to relaxation via decomposition. These problems comprise multiple subproblems that are independent of each other ...except for a collection of coupling constraints on the action space. We fit an additively separable value function approximation using two techniques, namely, Lagrangian relaxation and the linear programming (LP) approach to approximate dynamic programming. We prove various results comparing the relaxations to each other and to the optimal problem value. We also provide a column generation algorithm for solving the LP-based relaxation to any desired optimality tolerance, and we report on numerical experiments on bandit-like problems. Our results provide insight into the complexity versus quality trade-off when choosing which of these relaxations to implement.
In this paper, a novel iterative Q-learning method called "dual iterative Q-learning algorithm" is developed to solve the optimal battery management and control problem in smart residential ...environments. In the developed algorithm, two iterations are introduced, which are internal and external iterations, where internal iteration minimizes the total cost of power loads in each period, and the external iteration makes the iterative Q-function converge to the optimum. Based on the dual iterative Q-learning algorithm, the convergence property of the iterative Q-learning method for the optimal battery management and control problem is proven for the first time, which guarantees that both the iterative Q-function and the iterative control law reach the optimum. Implementing the algorithm by neural networks, numerical results and comparisons are given to illustrate the performance of the developed algorithm.
The prospective spread of electric vehicles (EV) and plug-in hybrid EV raises the need for fast charging rates. High required charging rates lead to high power demands, which may not be supported by ...the grid. In this paper, an optimal power flow technique of a PV-battery powered fast EV charging station is presented to continuously minimize the operation cost. The objective is to help the penetration of PV-battery systems into the grid and to support the growing need of fast EV charging. An optimization problem is formulated along with the required constraints and the operating cost function is chosen as a combination of electricity grid prices and the battery degradation cost. In the first stage of the proposed optimization procedure, an offline particle swarm optimization (PSO) is performed as a prediction layer. In the second stage, dynamic programming (DP) is performed as an online reactive management layer. Forecasted system data is utilized in both stages to find the optimal power management solution. In the reactive management layer, the outputs of the PSO are used to limit the available state trajectories used in the DP and, accordingly, improve the system computation time and efficiency. Online error compensation is implemented into the DP and fed back to the prediction layer for necessary prediction adjustments. Simulation and 1 kW prototype experimental results are successfully implemented to validate the system effectiveness and to demonstrate the benefits of using a hybrid grid tied system of PV-battery for fast EVs charging stations.
This paper develops an event-triggered decentralized tracking control (DTC) approach for modular reconfigurable robots (MRRs) by using adaptive dynamic programming. By establishing a decentralized ...neural network (NN) observer, which uses local input-output data and desired states of coupling subsystems, the local dynamics of MRR subsystem can be obtained. In order to obtain the DTC, the tracking error subsystem is augmented by the exosystem with the desired trajectory. Based on the event-triggered mechanism and a modified local cost function, the DTC is derived by solving the local Hamilton-Jacobi-Bellman equation via a local critic NN with asymptotically stable structure. The stability of the entire closed-loop MRR system is analyzed by Lyapunov's direct method. The simulation of a two-degree of freedom MRR system ensures that the developed event-triggered DTC scheme is effective.
This paper is concerned with a new iterative thetas -adaptive dynamic programming (ADP) technique to solve optimal control problems of infinite horizon discrete-time nonlinear systems. The idea is to ...use an iterative ADP algorithm to obtain the iterative control law which optimizes the iterative performance index function. In the present iterative thetas -ADP algorithm, the condition of initial admissible control in policy iteration algorithm is avoided. It is proved that all the iterative controls obtained in the iterative thetas -ADP algorithm can stabilize the nonlinear system which means that the iterative thetas -ADP algorithm is feasible for implementations both online and offline. Convergence analysis of the performance index function is presented to guarantee that the iterative performance index function will converge to the optimum monotonically. Neural networks are used to approximate the performance index function and compute the optimal control policy, respectively, for facilitating the implementation of the iterative thetas -ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the established method.
In this paper we discuss statistical properties and convergence of the Stochastic Dual Dynamic Programming (SDDP) method applied to multistage linear stochastic programming problems. We assume that ...the underline data process is stagewise independent and consider the framework where at first a random sample from the original (true) distribution is generated and consequently the SDDP algorithm is applied to the constructed Sample Average Approximation (SAA) problem. Then we proceed to analysis of the SDDP solutions of the SAA problem and their relations to solutions of the “true” problem. Finally we discuss an extension of the SDDP method to a risk averse formulation of multistage stochastic programs. We argue that the computational complexity of the corresponding SDDP algorithm is almost the same as in the risk neutral case.
Dynamic Energy Management System for a Smart Microgrid Venayagamoorthy, Ganesh Kumar; Sharma, Ratnesh K.; Gautam, Prajwal K. ...
IEEE transaction on neural networks and learning systems,
2016-Aug., 2016-08-00, 2016-8-00, 20160801, Volume:
27, Issue:
8
Journal Article
Open access
This paper presents the development of an intelligent dynamic energy management system (I-DEMS) for a smart microgrid. An evolutionary adaptive dynamic programming and reinforcement learning ...framework is introduced for evolving the I-DEMS online. The I-DEMS is an optimal or near-optimal DEMS capable of performing grid-connected and islanded microgrid operations. The primary sources of energy are sustainable, green, and environmentally friendly renewable energy systems (RESs), e.g., wind and solar; however, these forms of energy are uncertain and nondispatchable. Backup battery energy storage and thermal generation were used to overcome these challenges. Using the I-DEMS to schedule dispatches allowed the RESs and energy storage devices to be utilized to their maximum in order to supply the critical load at all times. Based on the microgrid's system states, the I-DEMS generates energy dispatch control signals, while a forward-looking network evaluates the dispatched control signals over time. Typical results are presented for varying generation and load profiles, and the performance of I-DEMS is compared with that of a decision tree approach-based DEMS (D-DEMS). The robust performance of the I-DEMS was illustrated by examining microgrid operations under different battery energy storage conditions.
![A Novel Iterative [thetas] ...](http://api.summon.serialssolutions.com/2.0.0/image/issn/XR4PS5YY4K/1545-5955/small "A Novel Iterative [thetas] -Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems")
PDF
