Wind energy technologies have been investigated extensively due to worldwide environmental challenges and rising energy demand. Therefore, accurate and reliable wind speed forecasts are essential for ...large scale wind power integration. But seasonal and stochastic winds make forecasting difficult. Hence, this study proposes a novel loss function-based hybrid deep learning architecture for the wind speed forecasting (WSF). The proposed hybrid model is developed using the improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) decomposition method for the denoising wind speed data, and transformer network (TRA) with a novel Kernel MSE loss function (NLF) for the WSF (NLF-ICEEMDAN-TRA). As the MSE loss function is sensitive to outliers and fails to identify the non-linear characteristics in wind speed data, a novel kernel MSE loss function has been used for the training of the transformer network. Wind speed data from two wind farms located in Block Island and the Gulf Coast have been used to validate the effectiveness of the proposed hybrid model. Because current WSF methods performance declines as the time ahead rises, the proposed hybrid model is verified using eight time horizons: 5 min, 10 min, 15 min, 30 min, 1 h, 2 h, 24 h and 48 h ahead WSF. To investigate the performance of proposed hybrid model in WSF, six individual WSF models, six hybrid WSF models, and six NLF based hybrid WSF models are employed for comparative analysis. The experimental results demonstrated that the proposed hybrid model achieved the best results for all eight time horizon WSF for both wind farm sites, with a significant improvement.
•Proposed approach is developed using ICEEMDAN and transformer model with novel loss function.•The improved CEEMDAN algorithm is used for denoising wind speed data.•The performance is investigated using three sets of WSF models.•Proposed model gave best results for 5 min, 10 min, 15 min, 30 min, 1 h, 2 h, 24 h, 48 h ahead WSF.
•Proposing an AI enabled self-powered wireless sensing system for smart industry.•Obtaining the voltage of the products by the TENG powered flexible sensor.•Realizing the products recognition by TENG ...with the transformer model in AI.•Powering the wireless sensing system by EMG and TENG hybrid generator.•Improving the smart industry with low-carbon, green and sustainable.
Traditional batteries or external supply powered wireless sensing system are needed to be improved for realizing the development of the smart industry with low-carbon, green and sustainable. This paper proposes and develops a self-powered wireless sensing system for smart industry (SPOT), utilizing a triboelectric nanogenerator (TENG) coupled with an artificial intelligence (AI) transformer model. The SPOT system includes the TENG-based self-powered flexible sensor (SWNG), the wireless aggregate node (WAN), the electromagnetic and TENG hybrid generator (ETCG), and the monitoring and management center with an AI model (MACA). The ETCG serves as a power source for the WAN. The SWNG acquires voltage signals from products on the conveyor belt in the smart industry, powered by the TENG, and transmits the sensor data wirelessly to the MACA via the WAN for processing. The MACA processes the data using the transformer AI model, which not only ensures self-sustainability and long-term stability but also enables intelligent recognition and monitoring of industrial products by their packaging materials, thereby providing precise status information and decision support for the smart industry. The transformer model’s deployment in the MACA has demonstrated robustness and a high classification success rate of up to 97.8 %, efficiently categorizing multiple targets. Additionally, the SWNG and WAN exhibit low power consumption of approximately 80 mW, successfully contributing to the realization of green, low-carbon objectives. The SPOT system significantly enhances the efficiency of product transportation and management within the smart industry and contributes to the advancement of a sustainable, low-carbon, and green smart industry, offering novel technological insights and pathways for future development.
With the rapid development of artificial intelligence and sensor technology, electroencephalogram-based (EEG) emotion recognition has attracted extensive attention. Various deep neural networks have ...been applied to it and achieved excellent results in classification accuracy. Except for classification accuracy, the interpretability of the feature extraction process is also considerable for model design for emotion recognition. In this study, we propose a novel neural network model (DCoT) with depthwise convolution and Transformer encoders for EEG-based emotion recognition by exploring the dependence of emotion recognition on each EEG channel and visualizing the captured features. Then we conduct subject-dependent and subject-independent experiments on a benchmark dataset, SEED, which contains EEG data of positive, neutral, and negative emotions. For subject-dependent experiments, the average accuracy of three classification tasks is 93.83%. For subject-independent experiments, the average accuracy of three classification tasks is 83.03%. Additionally, we assess the importance of each EEG channel in emotional activities by the DCoT model and visualize it as brain maps. Furthermore, satisfactory results are obtained by utilizing eight selected crucial EEG channels: FT7, T7, TP7, P3, FC6, FT8, T8, and F8, both in two classification tasks and three classification tasks. Using a small number of EEG channels for emotion recognition can reduce equipment costs and computing costs, which is suitable for practical applications.
•We proposed a novel neural network based on Transformer encoders and depthwise convolution for EEG-based emotion recognition.•Our model enhances the capability of extracting and fusing frequency-domain information while maintaining the independence between EEG channels.•The crucial EEG channels captured by the proposed model are visualized as brain maps in this work, and few existing works have done it.•Our model is much more interpretable than existing emotion recognition models by exploring the dependence of emotion recognition on each EEEG channel, which is useful in clinical applications.
•Pre-extraction mechanism is put forward to perform the feature construction.•Adaptive transformer is proposed for the complete degradation modeling.•An end-to-end deep framework is proposed for RUL ...prediction of bearings.•Two targeted case studies are designed to illustrate our superiority.
In practical engineering, accurate prediction of remaining useful life (RUL) is always necessary for effective preparation of engineering assets, human resources and maintenance actions. With the improvement of computing power and the passionate requirements for the high prediction accuracy of complex systems, more and more deep model-based frameworks have been developed for RUL prediction. In general, these frameworks consist of two stages: the first one is the manual operation of feature extraction and feature selection; the second one is the RUL prediction that mainly employs the recurrent deep models. However, such the frameworks do not fully take advantage of the deep models since they still rely on much prior knowledge and do not achieve the satisfied prediction performance. In this paper, a novel two stage framework with less prior knowledge, namely, end-to-end framework, is proposed to improve the forecasting performance. In our first stage, a feature pre-extraction mechanism is designed to pre-extract the low-level features in relatively high dimensional space, which requires no additional manual operations of feature fusion and feature selection in existing methods. In our second stage, adaptive transformer, a new deep model integrating the attention mechanism and the recurrent architecture, is proposed to model the relationships between these low-level features and the RULs directly, which suppresses the issue of vanishing gradients and is more suitable for representing the complex temporal degradation characteristics. Two public bearing datasets are employed to validate the effectiveness of the proposed framework in this paper. In these two case studies, some existing state-of-the-art RUL prediction approaches are fully compared, and the critical hyperparameters and components of our framework are analyzed in details. The experimental results reveal our advantage on adaptive degradation modeling and accurate RUL prediction, and help to interpret the impact of the proposed framework architecture on bearing RUL prediction.
The time series is generally a set of random variables that are observed and collected at a certain frequency in the course of something??s development. The task of time series forecasting is to ...extract the core patterns from a large amount of data and to make accurate estimates of future data based on known factors. Due to the access of a large number of IoT data collection devices, the explosive growth of multidimensional data and the increasingly demanding requirements for prediction accuracy, it is difficult for classical parametric models and traditional machine learning algorithms to meet high efficiency and high accuracy requirements of prediction tasks. In recent years, deep learning algorithms represented by convolutional neural networks, recurrent neural networks and Trans-former models have achieved fruitful results in time series forecasting tasks. To further promote the development of time series prediction technology, common characteristics of time series data, evaluation indexes of datasets and models are reviewed, and the characteristics, advantages and limitations of each prediction algorithm are experimentally compared and analyzed with time and algorithm architecture as the main research line. Several time series prediction methods based on Transformer model are highlighted and compared. Finally, according to the problems and challenges of deep learning applied to time series prediction tasks, this paper provides an outlook on the future research trends in this direction.
•A probabilistic forecasting method with interpretability and low temporal resolution for mid-term hourly load time series is proposed.•Model complexity is reduced by reconstructing raw univariate ...load time series to preserve the day-to-day load differences while shortening the time series.•It can explain how much each input feature influences the load at different times of the day.•A novel quantile loss function with quantile constraints and PI penalty significantly improves the accuracy of ITFT by increasing the reliability and sharpness of PIs.
The growth of distributed renewable energy and demand-side responsiveness has increased the difficulty of mid-term hourly load time-series forecasting. This study presents a probabilistic forecasting method for hourly load time series based on an improved temporal fusion transformer (ITFT) model to achieve more accurate and thorough forecasting results. The raw univariate time series of the hourly load was reconstructed into multiple day-to-day load time series at different hour-points to reconcile the contradiction between learning the temporal dependence on a long prediction horizon and reducing model complexity. The corresponding hour point was used as a static covariable input to distinguish the differences. Based on the original temporal fusion transformer (TFT) model, the ITFT model replaces the long short-term memory (LSTM) with a gated recurrent unit (GRU) to learn long-term dependence more efficiently. Furthermore, quantile constraints and prediction interval (PI) penalty terms were incorporated into the original quantile loss function to prevent quantile crossover and construct more compact prediction intervals (PIs). The results of two actual examples show that the proposed method is explanatory and can significantly improve the reliability and compactness of probabilistic load forecasting (PLF) results compared with other popular methods.
Granulation is a common internal disease in citrus fruits, and it is difficult to distinguish fruits with granulation disease from their appearance. In this study, a novel acoustic vibration device ...based on a micro-LDV, a microphone and a resonance speaker was employed to collect acoustic vibration response signals of "Aiyuan 38" jelly orange. The one-dimensional acoustic vibration response signal was converted into acoustic vibration images, and a double-input Resnet-Transformer network (DresT) was constructed for extracting deep features in acoustic vibration images for identifying jelly-orange granulation disease. Firstly, train Drest and Resnet50 models using acoustic vibration images and compare the performance of Drest with that of Resnet50 (based on CNN). Then PLS-DA and SVM models are trained using acoustic vibration image texture features or acoustic vibration spectral features, and the performance is compared with the DresT model. The results showed that the DresT model trained using acoustic vibration images can accurately identify jelly orange granulation disease with a detection accuracy of 99.31 %. The F1 of the model is 99.5 %, the accuracy is 99.01 %, and the recall is 100 %.
•Construct a dual-input Resnet-Transformer network (DresT).•Detection of jelly orange granulation disease using a novel acoustic vibration device.•Converted acoustic vibration response signals to acoustic vibration images.•Compared performance of models based on different types of data.•Achieved precise identification of Jelly Orange granulation disease.
•A Transformer model is applied to evaluate crash risk based on non-aggregated abnormal driving events (ADEs).•The proposed Transformer model outperforms other commonly used methods.•Impacts of the ...acceleration, speed, duration, and type of ADEs on crash risk are quantified.•A time-decay function is proposed to fit the temporal impacts of ADEs on crash risk.•Crash risk spatial–temporal decay and collective superposition effect of multiple ADEs are revealed.
A crash risk evaluation model aims to estimate crash occurrence possibility by establishing the relationships between traffic flow status and crash occurrence. Based upon which, Proactive Traffic Safety Management (PTSM) systems have been developed and implemented. The current crash risk evaluation models relied on high dense traffic detectors, which limited the applications of PTSM to infrastructures with enough sensing devices. To address such application limitation issue, this study employed the widespread abnormal driving event information that is generated by emerging driving monitoring and vehicle connection techniques to develop the crash risk evaluation model. Specifically, to characterize abnormal driving events, a six-tuple embedding method was proposed to store their space, time and kinetics features. Given their irregular and discrete distributions on roadways, a Transformer model with self-attention mechanism was proposed to extract the spatial distribution characteristics. In addition, a time-decay function was integrated to fit the temporal impacts of abnormal driving events on crash risk. Empirical data from a freeway in China were utilized for the analyses. The results showed that abnormal driving events with lower speed, larger acceleration and duration are more likely to cause crashes. The accumulation of multiple events in the time period of less than 3 min would lead to a sharp increase of crash risk. Besides, compared to the average metrics of the widely adopted Convolutional Neural Network (CNN), XGBoost, and logistic regression models, the proposed model achieved higher accuracy (0.841) and AUC (0.777), with average improvement of 2.5 % and 9.1 % respectively.
