Spot-Adaptive Knowledge Distillation Song, Jie; Chen, Ying; Ye, Jingwen ...
IEEE transactions on image processing,
2022, Volume:
31
Journal Article
Peer reviewed
Open access
Knowledge distillation (KD) has become a well established paradigm for compressing deep neural networks. The typical way of conducting knowledge distillation is to train the student network under the ...supervision of the teacher network to harness the knowledge at one or multiple spots ( i.e. , layers) in the teacher network. The distillation spots, once specified, will not change for all the training samples, throughout the whole distillation process. In this work, we argue that distillation spots should be adaptive to training samples and distillation epochs. We thus propose a new distillation strategy, termed spot-adaptive KD (SAKD), to adaptively determine the distillation spots in the teacher network per sample, at every training iteration during the whole distillation period. As SAKD actually focuses on "where to distill" instead of "what to distill" that is widely investigated by most existing works, it can be seamlessly integrated into existing distillation methods to further improve their performance. Extensive experiments with 10 state-of-the-art distillers are conducted to demonstrate the effectiveness of SAKD for improving their distillation performance, under both homogeneous and heterogeneous distillation settings. Code is available at https://github.com/zju-vipa/spot-adaptive-pytorch .
Membrane distillation (MD) is a promising separation technique capable of being used in the desalination of marine and brackish water. Poly(vinylidene fluoride) (PVDF) flat sheet nano-composite ...membranes were surface modified by coating with electro-spun PVDF nano-fibres to increase the surface hydrophobicity. For this purpose, the nano-composite membrane containing 7 wt.% superhydrophobic SiO2 nano-particles, which showed the highest flux in our previous work, was first subjected to pore size augmentation by increasing the concentration of the pore forming agent (Di-ionized water). Then, the prepared flat sheet membranes were subjected to nanofibres coating by electro-spinning. The uncoated and coated composite fabricated membranes were characterized using contact angle, liquid entry pressure of water, and scanning electron microscopy. The membranes were further tested for 6 h desalination by direct contact membrane distillation (DCMD) and vacuum membrane distillation (VMD), with a 3.5 wt.% synthetic NaClaq as the feed. In DCMD the feed liquid and permeate side temperature were maintained at 27.5 °C and 15 °C, respectively. For VMD, the feed liquid temperature was 27 °C and a vacuum of 94.8 kPa was applied on the permeate side. The maximum permeate flux achieved was 3.2 kg/m2.h for VMD and 6.5 kg/m2.h for DCMD. The salt rejection obtained was higher than 99.98%. The coated membranes showed a more stable flux than the uncoated membranes indicating that the double layered membranes have great potential in solving the pore wetting problem in MD.
Display omitted
•PVDF/SiO2 nano-composite membranes were prepared by phase inversion.•PVDF nano-fibres were electro-spun on the nano-composite membrane to form a dual layer membrane.•The dual layer membrane showed increased LEPw and contact angle.•The double layer membrane presented high tendencies to prevent pore wetting.•The DCMD flux was greater than the VMD flux for desalination using the dual layer membrane.
Membrane Distillation (MD) is based on the evaporation of a hot feed through a microporous and hydrophobic membrane. Thermal energy is needed to heat the feed up to the desired temperature during the ...process. Therefore, improvements of the thermal performance of MD are important to apply this technology at large scale. The Special Issue “Thermal Performance of Membrane Distillation” focuses on the recent research efforts made in this direction, covering both the development of new membranes and the optimization of the process.
Thermoplasmonic effects notably improve the efficiency of vacuum membrane distillation, an economically sustainable tool for high‐quality seawater desalination. Poly(vinylidene fluoride) (PVDF) ...membranes filled with spherical silver nanoparticles are used, whose size is tuned for the aim. With the addition of plasmonic nanoparticles in the membrane, the transmembrane flux increases by 11 times, and, moreover, the temperature at the membrane interface is higher than bulk temperature.
Due to the limited availability of freshwater supplies, desalination has become an increasingly reliable process for water supply worldwide, with proved technical and economic feasibility and ...advantages. Recently, desalination capacity significantly increased from approximately 35 million m3 daily (MCM/day) in 2005 to about 95 MCM/day in 2018. Seawater desalination accounts for about 61% of global desalination capacity, while brackish water desalination accounts for 30%. Membrane desalination, mainly using reverse osmosis (RO), accounts for ¾ of global desalination capacity, with the rest mostly used for thermal desalination using multi-stage flash distillation (MSF), and multi-effect distillation (MED). Despite the undeniable role of desalination for securing water supply in areas where natural freshwater supplies are scarce, desalination impacts the natural environment at different aspects. Environmental impacts (EIs) of the desalination process are different and vary significantly according to the nature of the utilized feedwater, the desalination technology in use, and the management of waste brine generated. In this work, the EIs of each desalination technology were thoroughly investigated, with careful consideration given to different feedwater qualities, and various brine management techniques. Although the different aspects of desalination EIs have been extensively studied in the literature, the literature lacks comprehensive reviews and summaries of all the associated EIs. This article compiles the different EIs associated with the whole desalination process in one-hub, applying an intake-to-outfall approach. The leading desalination technologies of RO, MSF, and MED were analyzed, along with different feedwaters. This article provides a mapping of the different technologies involving feedwater and brine management techniques and a detailed description of their impact on the environment. Finally, recommendations and conclusions were given to minimize the negative impacts of desalination on both the local and global environments.
Display omitted
•Demand for water supply is increasing worldwide with limited fresh water resources.•Desalination is becoming a more reliable non-conventional water supply.•Desalination processes are technology and energy intensive processes.•Desalination process results in air, marine, land pollution and should be minimized.•Environmental impacts of desalination processes have to be thoroughly considered.
A simple approach is introduced to locate a side‐draw tray for ternary and multi‐component mixtures with middle boiling component(s) present in the system at trace levels. The concept is based on a ...probability function defined by the thermodynamic properties of the system. The advantage of this method over existing methods is the ability to quickly and efficiently provide a feasible configuration of the distillation unit without relying on rigorous optimization or trial and error approaches. Moreover, it provides an intuitive understanding of the movements of the middle boiling components in the column.
By-products of essential oils (EOs) in the industry represent an exploitable material for natural and safe antioxidant production. One representative group of such by-products is distilled solid ...residues, whose composition is properly modulated by the distillation method applied for the recovery of EOs. Recently, in terms of Green Chemistry principles, conventional extraction and distillation processes are considered outdated and tend to be replaced by more environmentally friendly ones. In the present study, microwave-assisted hydro-distillation (MAHD) was employed as a novel and green method for the recovery of EOs from three aromatic plants (rosemary, Greek sage and spearmint). The method was compared to conventional ones, hydro-distillation (HD) and steam-distillation (SD), in terms of phytochemical composition of distilled solid residues, which was estimated by spectrophotometric and chromatographic methods. Total phenolic content (TPC), total flavonoid content (TFC) and antioxidant activity (ABTS, DPPH and FRAP) results highlighted the distilled solid residues as good sources of antioxidants. Moreover, higher antioxidant activity was achieved for MAHD extracts of solid residues in comparison to HD and SD extracts. A metabolomics approach was carried out on the methanolic extracts of solid residues obtained by different distillation methods using LC-MS analysis followed by multivariate data analysis. A total of 29 specialized metabolites were detected, and 26 of them were identified and quantified, presenting a similar phenolic profile among different treatments, whereas differences were observed among different species. Rosmarinic acid was the most abundant phenolic compound in all extracts, being higher in MAHD extracts. In rosemary and Greek sage extracts, carnosol and carnosic acid were quantified in significant amounts, while trimers and tetramers of caffeic acid (salvianolic acids isomers) were identified and quantified in spearmint extracts, being higher in MAHD extracts. The obtained results pointed out that MAHD extracts of distilled solid by-products could be a good source of bioactives with potential application in the food, pharmaceutical and cosmetic industries, contributing to the circular economy.
Deep learning techniques have largely solved the problem of rail surface defect detection (SDD), however, two aspects have yet to be addressed. In most existing approaches, two red-green-blue and ...depth (RGB-D) streams are indiscriminately fused across modalities, ignoring the fact that RGB and depth images produce different feature qualities in different scenes. Additionally, in their focus on performance, previous studies have overlooked the fact that models produce several parameters, resulting in unrealistic practical applications. To address these challenges, we designed a modal evaluation network (MENet) via knowledge distillation (KD) (MENet-S*) for a no-service rail SDD to adaptively manage information in each scenario and achieve model compression. First, to dynamically adjust the feature distribution and quality, dynamic and static feature coding ideas are introduced. Second, modal evaluation distillation is introduced, which allows a compact model (MENet-S) to learn the feature evaluation process of a complex model (MENet-T). Third, to enable MENet-S to learn the dynamic encoding process of MENet-T and to improve the feature representation of MENet-S, we propose accessible knowledge distillation. Furthermore, multitiered KD is introduced to facilitate the learning of MENet-S. Based on extensive experiments using the industrial RGB-D dataset NEU RSDDS-AUG, we observed that MENet-S* (MENet-S with KD) outperformed 16 state-of-the-art methods. In addition, to demonstrate the generalization capability of MENet-S*, we evaluated the proposed network on three additional public datasets, and MENet-S* achieved competitive results. The source codes and results are available at https://github.com/hjklearn/MENet-KD .
Display omitted
► A review on current industrial dividing wall column applications. ► A review on current research activities in the area of dividing wall column technology. ► An analysis of current ...dividing wall columns for azeotropic, extractive and reactive distillation. ► A review on control strategies for dividing wall columns.
In spite of being an energy intensive process, distillation remains the most important separation method in the chemical process industry. Especially for the separation of mixtures with three or more components, the total energy requirement and the capital cost are very high. In this respect, dividing wall columns (DWCs) represent a very promising technology allowing a significant energy requirement reduction. This article reviews current industrial applications of DWCs and related research activities, including column configuration, design, modelling and control issues. Furthermore, the application of DWCs for azeotropic, extractive and reactive distillation is highlighted.
PDF
