Deploying renewable energy and implementing smart energy management strategies are crucial for decarbonizing Building Energy Systems (BES). Despite recent advancements in data-driven Deep ...Reinforcement Learning (DRL) for BES optimization, significant challenges still exist, such as the time-consuming and data-intensive nature of training DRL controllers and the complexity of environment dynamics in Multi-Agent Reinforcement Learning (MARL). Consequently, these obstacles impede the synchronization and coordination of multiple agent control, leading to slow DRL convergence performance. To address these issues. This paper proposes a novel approach to optimize hybrid building energy systems. We introduce an integrated system combining a multi-stage Proximal Policy Optimization (PPO) on-policy framework with Imitation Learning (IL), interacting with the model environment. To improve scalability and robustness of Multi-agent Systems (MAS), this approach is designed to enhance training efficiency with centralized training and decentralized execution. Simulation results of case studies demonstrate the effectiveness of the Multi-agent Deep Reinforcement Learning (MADRL) model in optimizing the operations of hybrid building energy systems in terms of indoor thermal comfort and energy efficiency. Results show the proposed framework significantly improve performance in achieving convergence in just 50 episodes for dynamic decision-making. The scalability and robustness of the proposed model have been validated across various scenarios. Compared with the baseline during cold and warm weeks, the proposed control approach achieved improvements of 34.86% and 46.10% in energy self-sufficiency ratio, respectively. Additionally, the developed MADRL effectively improved solar photovoltaic (PV) self-consumption and reduced household energy costs. Notably, it increased the average indoor temperature closer to the desired set-point by 1.33 °C, and improved the self-consumption ratio by 15.78% in the colder week and 18.47% in the warmer week, compared to baseline measurements. These findings highlight the advantages of the multi-stage PPO on-policy framework, enabling faster learning and reduced training time, resulting in cost-effective solutions and enhanced solar PV self-consumption.
•Integrated a multi-stage RL algorithm with imitation learning in the MAPPO framework to optimize system energy performances.•Proposed model achieved faster MAPPO convergence under resource constraints•Introduced a shared reward function in MAPPO to align agent incentives with collective energy optimization.•Improved solar PV self-consumption and reduced energy costs while maintaining indoor thermal comfort with MAPPO.•Validated the scalability and adaptability of MAPPO across various scenarios.
•Dispatching of inflexible generation/demand using storage is discussed.•Novel stochastic robust optimization formulation yields reliable schedules.•Consideration of probabilistic forecasts of ...inflexible power and energy profiles.•Simulations underpin achievable performance improvements.
Electric energy generation from renewable energy sources is generally non-dispatchable due to its intrinsic volatility. Therefore, its integration into electricity markets and in power system operation is often based on volatility-compensating energy storage systems. Scheduling and control of this kind of coupled systems is usually based on hierarchical control and optimization. On the upper level, one solves an optimization problem to compute a dispatch schedule and a coherent allocation of energy reserves. On the lower level, one performs online adjustments of the dispatch schedule using, for example, model predictive control. In the present paper, we propose a formulation of the upper level optimization based on data-driven probabilistic forecasts of the power and energy output of the uncontrollable loads and generators dependent on renewable energy sources. Specifically, relying on probabilistic forecasts of both power and energy profiles of the uncertain demand/generation, we propose a novel framework to ensure the online feasibility of the dispatch schedule with a given security level. The efficacy of the proposed scheme is illustrated by simulations based on real household production and consumption data.
The number of electric vehicles (EVs) has increased sharply with the start of the 21st century. Although the automobile industry believes that drivetrains equipped with two-speed transmission can ...increase the energy efficiency of EVs by 4–5% and improve other vehicle performance indicators, most EVs still use fixed-speed ratio reducers. Does the two-speed transmission make sense? What is the latest development in the industry? What are the reasons this has not yet been popularized? This article attempts to analyze the current situation in-depth and provide a forecast of technology development trends. First, the authors analyze the advantages of the two-speed transmission from the energy consumption, economy, dynamic performance, and other indicators of EVs. Second, the topology optimization of the non-power-interruption two-speed transmission is introduced, and many efforts have been made to analyze the numerous studies both in academic research and industrial development. Then, a unified index for the necessity of adopting a two-speed transmission in EVs is given, which is verified by numerous data of different passenger cars and commercial vehicles. Finally, the authors predict that two-speed transmissions will be used first in sports cars, luxury cars, and electric logistics vehicles within several years, and then gradually penetrate ordinary family cars.
•Analyze the advantages of the two-speed transmission of electric vehicles.•Introduce the topology optimization of two-speed transmission.•Give a unified index for the necessity of adopting a two-speed transmission.•Make predictions on the development trend of two-speed transmissions.
The liner shipping schedules determine the container transportation time and the arrival time of ships, which has a significant influence on the shipper selection behavior and the transportation ...demand. This paper addresses the container liner shipping schedule optimization with shipper selection behavior considered. Our problem is formulated as a mixed-integer nonlinear programming model, where the shipper selection behavior is evaluated by a nested logit model. A particle swarm optimization (PSO) framework embedded with CPLEX solver is designed, by combining the constraint relaxations and the linearization techniques with the heuristic rules. The numerical experiments are conducted based on the Persian Gulf route of COSCO SHIPPING LINES. The results show that: the total freight demand is increased by 23% and the weekly operation revenue is increased by 31% after considering shipper selection. Besides, we find that the planned ship speed should be increased for time-preference shippers with electronic or refrigerated products, while it should be decreased for price-preference shippers with general or bulk cargoes. These conclusions can provide decision support for the operation practice of liner shipping schedule design.
Parallel tempering (PT) methods are a popular class of Markov chain Monte Carlo schemes used to sample complex high‐dimensional probability distributions. They rely on a collection of N interacting ...auxiliary chains targeting tempered versions of the target distribution to improve the exploration of the state space. We provide here a new perspective on these highly parallel algorithms and their tuning by identifying and formalizing a sharp divide in the behaviour and performance of reversible versus non‐reversible PT schemes. We show theoretically and empirically that a class of non‐reversible PT methods dominates its reversible counterparts and identify distinct scaling limits for the non‐reversible and reversible schemes, the former being a piecewise‐deterministic Markov process and the latter a diffusion. These results are exploited to identify the optimal annealing schedule for non‐reversible PT and to develop an iterative scheme approximating this schedule. We provide a wide range of numerical examples supporting our theoretical and methodological contributions. The proposed methodology is applicable to sample from a distribution π with a density L with respect to a reference distribution π0 and compute the normalizing constant ∫Ldπ0. A typical use case is when π0 is a prior distribution, L a likelihood function and π the corresponding posterior distribution.
The objective of this research is to reduce energy consumption from intra airport shuttle operations by optimizing routes and schedules, without compromising on passenger travel experience. To ...achieve this objective, we propose an optimization model that generates optimal airport shuttle routes for a given set of constraints and a discrete-event simulator that evaluates the optimal shuttle routes in a stochastic environment to understand the tradeoffs between the amount of time passengers wait for shuttles, and shuttle energy consumption. The proposed optimization model and stochastic simulation are tested using shuttle route data provided by the Dallas Fort Worth International Airport. Results indicate that optimized routes can lead to a 20% energy reduction in shuttle operations with a modest 2-min increase in average shuttle wait times. The optimization model and simulator presented here are designed to be generalizable and can be adapted to optimize shuttle operations at any major airport.
This research effort contributes to the state-of-practice on airport shuttle route optimization by:•Focusing on within airport shuttle operations as opposed to shuttle operations connecting city centers and airports.•Providing a novel mathematical model for selecting routes, shuttles per route, and route shuttle size.•Integrating optimization with discrete-event simulation to explore the tradeoff between passenger travel time and energy use.
For reasons of tractability, the airline scheduling problem has traditionally been sequentially decomposed into various stages (e.g., schedule generation, fleet assignment, aircraft routing, and crew ...pairing), with the decisions from one stage imposed upon the decision making process in subsequent stages. Although this approach greatly simplifies the solution process, it unfortunately fails to capture the many dependencies between the various stages, most notably between those of aircraft routing and crew pairing, and how these dependencies affect the propagation of delays through the flight network. Because delays are commonly transferred between late running aircraft and crew, it is important that aircraft routing and crew pairing decisions are made together. The propagated delay may then be accurately estimated to minimize the overall propagated delay for the network and produce a robust solution for both aircraft and crew. In this paper we introduce a new approach to accurately calculate and minimize the cost of propagated delay in a framework that integrates aircraft routing and crew pairing.
•We propose dynamic infrastructure occupation to assess capacity under disturbances.•Dynamic infrastructure occupation assesses disturbed conditions with rescheduling.•The timetable compression ...method is extended to dynamic infrastructure occupation.•ROMA is extended to compute scheduled and dynamic infrastructure occupation.•Capacity consumption under disturbances improves considerable with ETCS L2.
This paper proposes the new concept of dynamic infrastructure occupation to assess infrastructure capacity under disturbed conditions as a complement to the established capacity indicator of scheduled infrastructure occupation. This new indicator is applied in a capacity assessment study of a Dutch railway corridor with different signalling configurations under both scheduled and disturbed traffic conditions. For scheduled conditions the standard UIC compression method for computing infrastructure occupation is used, while dynamic infrastructure occupation under disturbed conditions requires a Monte Carlo simulation set up. For the analysis we use the train dispatching system ROMA that combines the alternative graph formulation of train rescheduling with blocking time modelling of signalling constraints. For the disturbed conditions, four traffic control scenarios are considered: three heuristics and an advanced branch-and-bound algorithm. The results show that the scheduled infrastructure occupation with ETCS Level 2 significantly improves over the legacy Dutch NS’54/ATB. In delayed operations, there is a considerable gain for ETCS in terms of dynamic infrastructure occupation and punctuality compared to NS’54/ATB, since the braking distances decrease when delayed trains run at lower speeds, having a stabilizing effect on headway times, delay propagation and throughput.
•A new decision support tool for productivity improvement is introduced.•The delivery scheduling is optimized and presented via interactive visualization.•Delivery times are shifted in order to reach ...a minimum number of used vehicles.•We conduct an experimental study based on characteristics of real-world instances.•The results show an improvement of productivity with a better use of resources.
Nowadays there is more and more competition in the industrial sector. Globalization makes fierce rivalry in the market between the different stakeholders at all levels offering to the customers a wide choice of cheaper products. Therefore, it is crucial to adopt efficient strategies to do the right things better with less resources and more benefit. The choice of the best techniques and methods is important and often tools need to be implemented. The purpose of this work is to introduce how productivity can be improved through delivery schedule optimization based on a decision support tool. This work is driven from an industrial case study. The results show a productivity improvement with a better use of resources up to 10% and effortless logistics management. Moreover, a comparison study is conducted between Genetic Algorithm and Ant Colony Optimization showing that our approach outperforms them in efficiency (≈36% and ≈25% respectively) and in computation time.
In this work, we focus on the optimization of charging scheduling policy for shared electric vehicles (EVs) integrated with wind power generation. This problem is of significant importance nowadays ...because of the large adoption of EVs in modern societies and the increasing penetration of renewables. A particular challenge of the problem is the large action space, the size of which may increase exponentially with respect to the number of EVs in the system. This makes the problem difficult to solve in practice. A lot of efforts have been made to overcome the above difficulty. The previous study proposed least-laxity-longer-processing-time-first (LLLP) principle, a rule-based algorithm to schedule EVs' battery charging. The LLLP principle assigns higher priority to vehicles with less laxity and longer processing time. We extend the LLLP principle and further study the structural property of the charging scheduling problem. The main contributions in this work are as follows. First, we show that the LLLP applies to our problem and may be used to narrow down the action space while preserving the global optimality. Second, we provide a modified LLLP algorithm that may construct a policy in <inline-formula> <tex-math notation="LaTeX">O(NT) </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">N </tex-math></inline-formula> is the number of the EVs and <inline-formula> <tex-math notation="LaTeX">T </tex-math></inline-formula> is the number of time steps in the scheduling problem. Third, we use numerical experiments to show that the new algorithm performs better than other existing algorithms, including the least-laxity-shorter-processing-time-first (LLSP) principle, the earliest-deadline-first (EDF) principle, and the latest-deadline-first (LDF) principle. The new algorithm finds near-optimal policies (within 1% performance loss) and is at least 40 times faster than CPLEX. We hope that this work provides insight into the charging scheduling of shared EVs in general.
PDF
