•Reinforcement learning has been explored for building control applications.•We reviewed studies using reinforcement learning for building controls.•We surveyed algorithm, state, action, reward, and ...the environment for reinforcement learning controller.•Research trends, progress and gaps of this field have been identified.•Adoption of reinforcement learning based controls in real buildings still faces significant challenges.
Building controls are becoming more important and complicated due to the dynamic and stochastic energy demand, on-site intermittent energy supply, as well as energy storage, making it difficult for them to be optimized by conventional control techniques. Reinforcement Learning (RL), as an emerging control technique, has attracted growing research interest and demonstrated its potential to enhance building performance while addressing some limitations of other advanced control techniques, such as model predictive control. This study conducted a comprehensive review of existing studies that applied RL for building controls. It provided a detailed breakdown of the existing RL studies that use a specific variation of each major component of the Reinforcement Learning: algorithm, state, action, reward, and environment. We found RL for building controls is still in the research stage with limited applications (11%) in real buildings. Three significant barriers prevent the adoption of RL controllers in actual building controls: (1) the training process is time consuming and data demanding, (2) the control security and robustness need to be enhanced, and (3) the generalization capabilities of RL controllers need to be improved using approaches such as transfer learning. Future research may focus on developing RL controllers that could be used in real buildings, addressing current RL challenges, such as accelerating training and enhancing control robustness, as well as developing an open-source testbed and dataset for performance benchmarking of RL controllers.
The antimicrobial peptide database (APD, http://aps.unmc.edu/AP/) is an original database initially online in 2003. The APD2 (2009 version) has been regularly updated and further expanded into the ...APD3. This database currently focuses on natural antimicrobial peptides (AMPs) with defined sequence and activity. It includes a total of 2619 AMPs with 261 bacteriocins from bacteria, 4 AMPs from archaea, 7 from protists, 13 from fungi, 321 from plants and 1972 animal host defense peptides. The APD3 contains 2169 antibacterial, 172 antiviral, 105 anti-HIV, 959 antifungal, 80 antiparasitic and 185 anticancer peptides. Newly annotated are AMPs with antibiofilm, antimalarial, anti-protist, insecticidal, spermicidal, chemotactic, wound healing, antioxidant and protease inhibiting properties. We also describe other searchable annotations, including target pathogens, molecule-binding partners, post-translational modifications and animal models. Amino acid profiles or signatures of natural AMPs are important for peptide classification, prediction and design. Finally, we summarize various database applications in research and education.
•Building load prediction informs chiller plant and thermal storage optimization.•We used and compared 9 machine learning algorithms and 3 heuristic prediction methods.•XGBoost and LSTM are found to ...be the best shallow and deep learning algorithm.•LSTM is better for short term prediction, while XGBoost for long term prediction.•It is better to train the model with uncertain rather than accurate weather data.
Building thermal load prediction informs the optimization of cooling plant and thermal energy storage. Physics-based prediction models of building thermal load are constrained by the model and input complexity. In this study, we developed 12 data-driven models (7 shallow learning, 2 deep learning, and 3 heuristic methods) to predict building thermal load and compared shallow machine learning and deep learning. The 12 prediction models were compared with the measured cooling demand. It was found XGBoost (Extreme Gradient Boost) and LSTM (Long Short Term Memory) provided the most accurate load prediction in the shallow and deep learning category, and both outperformed the best baseline model, which uses the previous day’s data for prediction. Then, we discussed how the prediction horizon and input uncertainty would influence the load prediction accuracy. Major conclusions are twofold: first, LSTM performs well in short-term prediction (1 h ahead) but not in long term prediction (24 h ahead), because the sequential information becomes less relevant and accordingly not so useful when the prediction horizon is long. Second, the presence of weather forecast uncertainty deteriorates XGBoost’s accuracy and favors LSTM, because the sequential information makes the model more robust to input uncertainty. Training the model with the uncertain rather than accurate weather data could enhance the model’s robustness. Our findings have two implications for practice. First, LSTM is recommended for short-term load prediction given that weather forecast uncertainty is unavoidable. Second, XGBoost is recommended for long term prediction, and the model should be trained with the presence of input uncertainty.
Breast cancer radiation therapy cures many women, but where the heart is exposed, it can cause heart disease. We report a systematic review of heart doses from breast cancer radiation therapy that ...were published during 2003 to 2013.
Eligible studies were those reporting whole-heart dose (ie, dose averaged over the whole heart). Analyses considered the arithmetic mean of the whole-heart doses for the CT plans for each regimen in each study. We termed this "mean heart dose."
In left-sided breast cancer, mean heart dose averaged over all 398 regimens reported in 149 studies from 28 countries was 5.4 Gy (range, <0.1-28.6 Gy). In regimens that did not include the internal mammary chain (IMC), average mean heart dose was 4.2 Gy and varied with the target tissues irradiated. The lowest average mean heart doses were from tangential radiation therapy with either breathing control (1.3 Gy; range, 0.4-2.5 Gy) or treatment in the lateral decubitus position (1.2 Gy; range, 0.8-1.7 Gy), or from proton radiation therapy (0.5 Gy; range, 0.1-0.8 Gy). For intensity modulated radiation therapy mean heart dose was 5.6 Gy (range, <0.1-23.0 Gy). Where the IMC was irradiated, average mean heart dose was around 8 Gy and varied little according to which other targets were irradiated. Proton radiation therapy delivered the lowest average mean heart dose (2.6 Gy, range, 1.0-6.0 Gy), and tangential radiation therapy with a separate IMC field the highest (9.2 Gy, range, 1.9-21.0 Gy). In right-sided breast cancer, the average mean heart dose was 3.3 Gy based on 45 regimens in 23 studies.
Recent estimates of typical heart doses from left breast cancer radiation therapy vary widely between studies, even for apparently similar regimens. Maneuvers to reduce heart dose in left tangential radiation therapy were successful. Proton radiation therapy delivered the lowest doses. Inclusion of the IMC doubled typical heart dose.
Fenton reaction‐mediated chemodynamic therapy (CDT) can kill cancer cells via the conversion of H2O2 to highly toxic HO•. However, problems such as insufficient H2O2 levels in the tumor tissue and ...low Fenton reaction efficiency severely limit the performance of CDT. Here, the prodrug tirapazamine (TPZ)‐loaded human serum albumin (HSA)–glucose oxidase (GOx) mixture is prepared and modified with a metal–polyphenol network composed of ferric ions (Fe3+) and tannic acid (TA), to obtain a self‐amplified nanoreactor termed HSA–GOx–TPZ–Fe3+–TA (HGTFT) for sustainable and cascade cancer therapy with exogenous H2O2 production and TA‐accelerated Fe3+/Fe2+ conversion. The HGTFT nanoreactor can efficiently convert oxygen into HO• for CDT, consume glucose for starvation therapy, and provide a hypoxic environment for TPZ radical‐mediated chemotherapy. Besides, it is revealed that the nanoreactor can significantly elevate the intracellular reactive oxygen species content and hypoxia level, decrease the intracellular glutathione content, and release metal ions in the tumors for metal ion interference therapy (also termed “ion‐interference therapy” or “metal ion therapy”). Further, the nanoreactor can also increase the tumor’s hypoxia level and efficiently inhibit tumor growth. It is believed that this tumor microenvironment‐regulable nanoreactor with sustainable and cascade anticancer performance and excellent biosafety represents an advance in nanomedicine.
A self‐amplified nanoreactor termed HSA–GOx–TPZ–Fe3+–TA (HGTFT), consisting of human serum albumin, glucose oxidase, tirapazamine (TPZ), Fe3+, and tannic acid (TA), is prepared for sustainable and cascade cancer therapy with exogenous H2O2 production and TA‐accelerated Fe3+/Fe2+ conversion. The nanoreactor can convert oxygen into HO• for chemodynamic therapy, consume glucose for starvation therapy, and provide a hypoxic environment for TPZ radical‐mediated chemotherapy.
In this paper, we develop optimal energy scheduling algorithms for N-user fading multiple-access channels with energy harvesting to maximize the channel sum-rate, assuming that the side information ...of both the channel states and energy harvesting states for K time slots is known a priori, and the battery capacity and the maximum energy consumption in each time slot are bounded. The problem is formulated as a convex optimization problem with O (NK) constraints making it hard to solve using a general convex solver since the computational complexity of a generic convex solver becomes impractically high when the number of constraints is large. This paper gives an efficient energy scheduling algorithm, called the iterative dynamic water-filling algorithm, that has a computational complexity of O(NK 2 ) per iteration. For the single-user case, a dynamic water-filling method is shown to be optimal. Unlike the traditional water-filling algorithm, in dynamic water-filling, the water level is not constant but changes when the battery overflows or depletes. An iterative version of the dynamic water-filling algorithm is shown to be optimal for the case of multiple users. Even though in principle the optimality is achieved under large number of iterations, in practice convergence is reached in only a few iterations. Moreover, a single iteration of the dynamic water-filling algorithm achieves a sum-rate that is within (N-1)K nats of the optimal sum-rate.
The A‐site mixed‐ammonium solid solutions of metal–organic perovskites (NH2NH3)x(CH3NH3)1−xMn(HCOO)3 (x=1.00–0.67) exhibit para‐ to ferroelectric diffuse phase transitions with lowered transition ...temperatures from x=1.00 to 0.67. These properties are due to the decreased framework distortion and polarization in their low temperature ferroelectric phases caused by the increased CH3NH3+ concentration.
Mix and match: (NH2NH3)x(CH3NH3)1−xMn(HCOO)3 (x=1.00–0.67), an A‐site solid solution series of metal–organic perovskites, shows para‐ to ferroelectric transitions, with the critical temperature and phase‐transition character modulated by the ammonium composition. Phase transitions are studied by variable‐temperature X‐ray crystallography and dielectric permittivity (ε′) measurements (plots shown are for different ammonium compositions).
Background and Purpose
Neuropathic pain affects millions of patients, but there are currently few viable therapeutic options available. Microtubule affinity‐regulating kinases (MARKs) regulate the ...dynamics of microtubules and participate in synaptic remodelling. It is unclear whether these changes are involved in the central sensitization of neuropathic pain. This study examined the role of MARK1 or MARK2 in regulating neurosynaptic plasticity induced by neuropathic pain.
Experimental Approach
A rat spinal nerve ligation (SNL) model was established to induce neuropathic pain. The role of MARKs in nociceptive regulation was assessed by genetically knocking down MARK1 or MARK2 in amygdala and systemic administration of PCC0105003, a novel small molecule MARK inhibitor. Cognitive function, anxiety‐like behaviours and motor coordination capability were also examined in SNL rats. Synaptic remodelling‐associated signalling changes were detected with electrophysiological recording, Golgi‐Cox staining, western blotting and qRT‐PCR.
Key Results
MARK1 and MARK2 expression levels in amygdala and spinal dorsal horn were elevated in SNL rats. MARK1 or MARK2 knockdown in amygdala and PCC0105003 treatment partially attenuated pain‐like behaviours along with improving cognitive deficit, anxiogenic‐like behaviours and motor coordination in SNL rats. Inhibition of MARKs signalling reversed synaptic plasticity at the functional and structural levels by suppressing NR2B/GluR1 and EB3/Drebrin signalling pathways both in amygdala and spinal dorsal horn.
Conclusion and Implications
These results suggest that MARKs‐mediated synaptic remodelling plays a key role in the pathogenesis of neuropathic pain and that pharmacological inhibitors of MARKs such as PCC0105003 could represent a novel therapeutic strategy for the management of neuropathic pain.
Protein-protein interactions (PPIs) play an important role in the different functions of cells, but accurate prediction of the three-dimensional structures for PPIs is still a notoriously difficult ...task. In this study, HawkDock, a free and open accessed web server, was developed to predict and analyze the structures of PPIs. In the HawkDock server, the ATTRACT docking algorithm, the HawkRank scoring function developed in our group and the MM/GBSA free energy decomposition analysis were seamlessly integrated into a multi-functional platform. The structures of PPIs were predicted by combining the ATTRACT docking and the HawkRank re-scoring, and the key residues for PPIs were highlighted by the MM/GBSA free energy decomposition. The molecular visualization was supported by 3Dmol.js. For the structural modeling of PPIs, HawkDock could achieve a better performance than ZDOCK 3.0.2 in the benchmark testing. For the prediction of key residues, the important residues that play an essential role in PPIs could be identified in the top 10 residues for ∼81.4% predicted models and ∼95.4% crystal structures in the benchmark dataset. To sum up, the HawkDock server is a powerful tool to predict the binding structures and identify the key residues of PPIs. The HawkDock server is accessible free of charge at http://cadd.zju.edu.cn/hawkdock/.
•Proposing an improved YOLO-V3 network processed by DenseNet method.•Realizing the detection of apples in three different growth stages in orchards.•Realizing real-time detection of apples in high ...resolution images.•Realizing the detection of apples under occlusion and overlap conditions.
Real-time detection of apples in orchards is one of the most important methods for judging growth stages of apples and estimating yield. The size, colour, cluster density, and other growth characteristics of apples change as they grow. Traditional detection methods can only detect apples during a particular growth stage, but these methods cannot be adapted to different growth stages using the same model. We propose an improved YOLO-V3 model for detecting apples during different growth stages in orchards with fluctuating illumination, complex backgrounds, overlapping apples, and branches and leaves. Images of young apples, expanding apples, and ripe apples are initially collected. These images are subsequently augmented using rotation transformation, colour balance transformation, brightness transformation, and blur processing. The augmented images are used to create training sets. The DenseNet method is used to process feature layers with low resolution in the YOLO-V3 network. This effectively enhances feature propagation, promotes feature reuse, and improves network performance. After training the model, the performance of the trained model is tested on a test dataset. The test results show that the proposed YOLOV3-dense model is superior to the original YOLO-V3 model and the Faster R-CNN with VGG16 net model, which is the state-of-art fruit detection model. The average detection time of the model is 0.304s per frame at 3000 × 3000 resolution, which can provide real-time detection of apples in orchards. Moreover, the YOLOV3-dense model can effectively provide apple detection under overlapping apples and occlusion conditions, and can be applied in the actual environment of orchards.
PDF
