This paper proposes a data-driven cooperative method for load frequency control (LFC) of the multi-area power system based on multi-agent deep reinforcement learning (MA-DRL) in continuous action ...domain. The proposed method can nonlinearly and adaptively derive the optimal coordinated control strategies for multiple LFC controllers through centralized learning and decentralized implementation. The centralized learning is achieved by MA-DRL based on a global action-value function to quantify overall LFC performance of the power system. To solve the MA-DRL problem, multi-agent deep deterministic policy gradient (DDPG) is derived to adjust control agents' parameters considering the nonlinear generator behaviors. For implementation, each individual controller only needs local information in its control area to deliver optimal control signals. Numerical simulations on a three-area power system and the fully-modeled New-England 39-bus system demonstrate that the proposed method can effectively minimize control errors against stochastic frequency variations caused by load and renewable power fluctuations.
High-level penetration of intermittent renewable energy sources has introduced significant uncertainties and variabilities into modern power systems. In order to rapidly and economically respond to ...the changes in power system operating state, this letter proposes a real-time optimal power flow (RT-OPF) approach using Lagrangian -based deep reinforcement learning (DRL) in continuous action domain. A DRL agent to determine RT-OPF decisions is constructed and optimized using the deep deterministic policy gradient. The DRL action-value function is designed to simultaneously model RT-OPF objective and constraints. Instead of using the critic network, the deterministic gradient is derived analytically. The proposed method is tested on the IEEE 118-bus system. Compared with the state-of-the-art methods, the proposed method can achieve a high solution optimality and constraint compliance in real-time.
A battery energy storage system (BESS) is an effective solution to mitigate real-time power imbalance by participating in power system frequency control. However, battery aging resulted from ...intensive charge–discharge cycles will inevitably lead to lifetime degradation, which eventually incurs high-operating costs. This study proposes a deep reinforcement learning-based data-driven approach for optimal control of BESS for frequency support considering the battery lifetime degradation. A cost model considering battery cycle aging cost, unscheduled interchange price, and generation cost is proposed to estimate the total operational cost of BESS for power system frequency support, and an actor–critic model is designed for optimising the BESS controller performance. The effectiveness of the proposed optimal BESS control method is verified in a three-area power system.
This letter proposes a data-driven, model-free method for load frequency control (LFC) against renewable energy uncertainties based on deep reinforcement learning (DRL) in continuous action domain. ...The proposed method can nonlinearly derive control strategies to minimize frequency deviation with faster response speed and stronger adaptability for unmolded system dynamics. It consists of offline optimization of LFC strategies with DRL and continuous action search, and online control with policy network where features are extracted by stacked denoising auto-encoders. Numerical simulations verify the effectiveness and advantages of proposed method over existing approaches.
Wet ethanol is a biofuel that can be rapidly integrated into the existing transportation sector infrastructure and have an immediate impact on decarbonization. Compared to conventional hydrocarbon ...fuels, wet ethanol has unique fuel properties (e.g., short carbon chain, oxygenated, high heat of vaporization, no cool-flame reactivity), which can actually improve the efficiency and engine-out emissions of internal combustion engines while decarbonizing. In this work, wet ethanol 80 (80% ethanol, 20% water by mass) was experimentally studied at high loads under boosted conditions in compression ignition to study the tradeoffs in efficiency and emissions based on boosting and injection strategies. Specifically, this work explores the potential of adding a third, mixing-controlled injection at high loads. The results indicate that adding a third, mixing-controlled injection results in combustion stabilization at high loads, where the peak pressure limit of the engine is a constraint that requires combustion phasing to retard. However, since the heat of vaporization of wet ethanol 80 is ~6% of its lower heating value, evaporation of fuel injected near top dead center imposes a thermodynamic efficiency penalty by absorbing heat from the working fluid at a time in the cycle when adding heat produces net work out. Additionally, the mixing-controlled injection increases NOx emissions. Therefore, the amount of fuel injected in the mixing-controlled injection should be limited to only what is necessary to stabilize combustion. Ultimately, by using wet ethanol 80 in a triple injection strategy, a load of 22 bar IMEPn is achieved with a net fuel conversion efficiency of 42.2%, an engine-out indicated specific emissions of NOx of 1.3 g/kWh, and no measurable particulate matter, while maintaining a peak cylinder pressure below 150 bar.
This paper proposes a hybrid data-driven method for fast solutions of preventive security-constrained optimal power flow (SCOPF) of power systems. The proposed method formulates the SCOPF problem as ...constraints-satisfying training of a deep reinforcement learning (DRL) agent, where the action-value function of DRL is augmented by contingency security constraints. In the training process, the proposed method hybridizes the primal-dual deep deterministic policy gradient (PD-DDPG) and the classic SCOPF model. Instead of building reward critic networks and cost critic networks via interacting with the environment (i.e., power flow equations), the actor gradients are approximated by solving KKT conditions of the Lagrangian . Finally, with the formulated sparse Jacobians of constraints and sparse Hessians of Lagrangians, the interior point method is incorporated in PD-DDPG to derive the parameters updating rule of the DRL agent. Numerical tests are carried out on a modified IEEE 57-bus system and a modified IEEE 300-bus system for critical contingencies. The results show that the well-trained DRL agent can rapidly (real-time) obtain high-quality SCOPF solutions that satisfy the security constraints.
The tympanic membrane (TM), located at the end of the ear canal, is a collagenous multi-layer soft tissue membrane with fibers highly aligned in radial and circumferential orientations. This unique ...multi-layer fiber ultrastructure makes TM’s mechanical behavior display both anisotropy and nonlinearity, which is important in sound transmission. However, the constitutive model of TM which includes both features has not been proposed. In this study, we develop a fiber-reinforced mesoscale constitutive model of TM which captures both anisotropic and nonlinear elastic mechanical behaviors. The TM is considered a continuum fiber-reinforced composite with two families of collagen fibers. Its overall properties are built up by integrating its heterogeneous material properties through the thickness. The homogenized mechanical properties are assumed to be uniformly distributed through TM’s thickness and superposed by three uncoupled elastic contributions of radial collagen fibers, circumferential collagen fibers, and an equivalent isotropic matrix. The model is calibrated using literature data through the inverse method. Simulation results indicate that specific collagen fibers alignment is responsible for the significant spatial and directional variation of deformation of the TM strip. With the appropriate strength criteria related to fiber deformation, the anisotropic localized failure mode of the TM strip observed in the experiment can be captured. The nonlinear nature and rotation of collagen fiber bundles are the origin of the nonlinear mechanical behavior of TM strips under uniaxial loading. The mesoscale constitutive model offers a different perspective on TM’s anisotropic and nonlinear elastic mechanical behavior. This research improves our understanding of the mechanical behavior of the TM and could help biomimetic graft development.
Hypoxia in most solid tumors is a major challenge for photodynamic therapy (PDT), and the combination of hypoxia-activated chemotherapy and PDT is a promising approach for enhanced anticancer ...activity. Herein, we designed hypoxia-responsive polymeric nanoprodrug PNPs to co-deliver photosensitizer 5,10,5,20-tetrakis(4-aminophenyl)-porphine (TAPP) and chlorambucil (CB) to improve the overall therapeutic efficacy. Upon laser irradiation, the central TAPP converted oxygen to produce single oxygen (1O2) for PDT and induced PDT-reduced hypoxia environment, which accelerated the release of activated CB for synergetic cancer cell killing. Consequently, these hypoxia-responsive polymeric nanoprodrugs with a considerable drug-loading content and synergistic therapeutic effect of PDT-CT had great potential for tumor therapy.
