Predicting history-dependent materials’ responses is crucial, as path-dependent behavior appears while characterizing or geometrically designing many materials (e.g., metallic and polymeric cellular ...materials), and it takes place in manufacturing and processing of many materials (e.g., metal solidification). Such phenomena can be computationally intensive and challenging when numerical schemes such as the finite element method are used. Here, we have applied a variety of sequence learning models to almost instantly predict the history-dependent responses (stresses and energy) of a class of cellular materials as well as the multiphysics problem of steel solidification with multiple thermo-viscoplasticity constitutive models accounting for substantial temperature, time, and path dependencies, and phase transformation. We have shown the gated recurrent unit (GRU) as well as the temporal convolutional network (TCN), can both accurately learn and almost instantly predict these irreversible, and history- and time-dependent phenomena, while TCN is more computationally efficient during the training process. This work may open the door for the broader adoption of data-driven models in similar computationally challenging constitutive models in plasticity and inelasticity.
•Several sequence learning methods are applied and compared for path-dependent plasticity and thermo-viscoplasticity.•GRU and TCN sequence learning models accurately predict the complex behavior of history-dependent materials.•Once properly trained, these deep learning models can instantly inference good quality results for unseen input data.•This work opens the door to the broader adoption of data-driven models in studying computationally challenging materials.
Recently, the boom in wind power industry has called for the accurate and stable wind speed forecasting, on which reliable wind power generation systems depend heavily. Due to the intermittency and ...complexity of wind, an appropriate decomposition is proved as a pivotal part in the precise wind speed prediction. On this account, this paper constructs a hybrid decomposition method coupling the ensemble patch transform (EPT) and the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN), where EPT is utilized to extract the trend of wind speed, then CEEMDAN is employed to divide the volatility into several fluctuation components with different frequency characteristics. Subsequently, the proposed decomposition method is combined with temporal convolutional networks (TCN) for the individual prediction of the trend and fluctuation components. Ultimately, the forecasted values for the wind speed prediction are obtained by reconstructing the prediction results of all the components. To evaluate the performance of the proposed EPT-CEEMDAN-TCN model, the historical wind speed data from three wind farms across China are used. The experimental results verify the notable effectiveness and necessity of the proposed EPT-CEEMDAN decomposition. In the meanwhile, the results demonstrate the significant superiority of the proposed EPT-CEEMDAN-TCN model on accuracy and stability.
Display omitted
•The ensemble patch transformation (EPT) is established to extract the daily trend.•The CEEMDAN is applied to decompose the volatility of wind speed series.•The temporal convolutional networks (TCN) are adopted for prediction.•The EPT-CEEMDAN-TCN model is developed for multi-step-ahead wind speed forecasting.•The superiority of the proposed model is verified on datasets from diverse areas.
An accurate aging forecasting and state of health estimation is essential for a safe and economically valuable usage of lithium-ion batteries. However, the non-linear aging of lithium-ion batteries ...is dependent on various operating and environmental conditions wherefore the degradation estimation is a complex challenge. Moreover, for on-board estimations where only limited memory and computing power are available, a state of health estimation algorithm is needed that is able to process raw sensor data without complex preprocessing. This paper presents a data-driven state of health estimation algorithm for lithium-ion batteries using different segments of partial discharge profiles. Raw sensor data is directly input to a temporal convolutional neural network without the need of executing feature engineering steps. The neural network is able to process raw sensor data and estimate the state of health of battery cells for different aging and degradation scenarios. After executing Bayesian hyperparameter tuning together with a stratified cross validation approach for splitting the training and test data, the achieved generalized aging model estimates the state of health with an overall root mean squared error of 1.0%.
•Investigation of the influence of using partial load profiles on SOH Estimation.•SOH model that is able to process raw sensor data without preprocessing steps.•Generalized SOH model that accurately estimates the SOH for different aging histories.•Optimal model selection using Bayesian hyperparameter tuning.•Temporal convolutional neural network model with an overall SOH Estimation RMSE of 1%.
The unique nature of website URLs has made phishing detection a challenging task. Unlike natural language, URLs have an unstructured nature with non-linear and sophisticated correlations. Therefore, ...they should be handled as both natural language and unstructured data sequences. However, the current solutions for phishing URL detection only focused on a single aspect of web page URLs. In this concern, this paper proposes an integrated model based on DL classifiers and pre-trained transformer to examine both the unique nature and the natural language structure of URL sequences simultaneously. The proposed model consists of three modules: RasNet (Keras-ResNet), TCMA (TCN-MHSA), and MPNet (Masked and Permuted Pre-training for Language Understanding). Considering the unique nature of the input data, RasNet combines two Keras embedding techniques to obtain the feature representations of URLs and then fuses them using a Residual Network (ResNet) to balance the weight distribution among the character-level and word-level information. Additionally, TCMA integrates the Temporal Convolutional Network (TCN) with the Multi-Head Self-Attention (MHSA) mechanism to optimize feature extraction and improve classification accuracy. Concurrently, MPNet joins the advantages and eliminates the drawbacks of Masked Language Modelling and Permuted Language Modelling to examine the nature language structure of web page URLs. The proposed model was trained and tested on four different datasets, including Ebbu2017, PhishCrawl, 420K-PD, and 1M-PD. The experimental results indicated that the proposed solution outperformed other models in classifying malicious URLs with the highest detection rate of 99.71% on the 1M-PD dataset, improving the performance accuracy of the state-of-the-art approaches by 1.37% to 2.01%.
Display omitted
•This paper proposed an integrated model, called RasNet-TCMA-MPNet.•RasNet combines Keras embedding with ResNet to analyse the URL’s unique nature.•TCMA integrates TCN with MHSA to enhance feature selection and classification.•MPNet combines the strengths of MLM and PLM for effective detection of phishing URLs.•The proposed model outperformed all baseline architectures in four different datasets.
Multimodal sentiment analysis aims to extract and integrate information from different modalities to accurately identify the sentiment expressed in multimodal data. How to effectively capture the ...relevant information within a specific modality and how to fully exploit the complementary information among multiple modalities are two major challenges in multimodal sentiment analysis. Traditional approaches fail to obtain the global contextual information of long time-series data when extracting unimodal temporal features, and they usually fuse the features from multiple modalities with the same method and ignore the correlation between different modalities when modeling inter-modal interactions. In this paper, we first propose an Attentional Temporal Convolutional Network (ATCN) to extract unimodal temporal features for enhancing the feature representation ability, then introduce a Multi-layer Feature Fusion (MFF) model to improve the effectiveness of multimodal fusion, which fuses the different-level features by different methods according to the correlation coefficient between the features, and cross-modal multi-head attention is used to fully explore the potential relationship between the low-level features. The experimental results on SIMS and CMU-MOSI datasets show that the proposed model achieves superior performance on sentiment analysis tasks compared to state-of-the-art baselines.
Lithium-ion batteries (LIBs) need to maintain high energy efficiency and power level in several application scenario. Accurate state of health (SOH) forecast is essential for designing a safe and ...reliable battery management systems (BMS). Temporal convolutional network (TCN) is a prevailing deep learning method for estimating the SOH of lithium-ion batteries. However, the hyperparameters in the network are usually difficult to predefine, which poses a challenge for the SOH estimation accuracy in real-world. To solve this problem, this paper proposes a data-driven estimation approach, where the TCN is combined with the modified flower pollination algorithm (MFPA) to determine the network topology. After hyperparameter optimization, the external sensor raw data and identified ohmic resistances trajectories in the equivalent circuits model (ECM) are both input to the TCN model to estimate SOH of LIBs. In contrast to prior approaches for feature extraction, this method is not only conductive to improve SOH estimation accuracy, but also can reduce on-board estimation computing burden. We carry out experiments on the same type of cells from NASA public data resources. The experimental results systematically validate the superiority of the proposed method, which covers high estimation accuracy, great robustness to varied training set and satisfied universality to different batteries.
Display omitted
•The MFPA algorithm is introduced to optimize several key hyper parameters in the TCN structure•Extracting external morphological features from the raw voltage and current curves•Ohmic resistances trajectories in the ECM with aging mechanisms to improve the SOH estimation accuracy•High estimation accuracy, great robustness to varied training set and satisfied universality to different battery types
Vascular interventional surgery (VIS) robot is a surgical treatment plan that effectively protects surgeons from X-ray radiation. However, the master-slave control method cuts off the surgeons' ...natural force feedback, which increases the risk of surgical safety. Most VIS robotic systems use force sensors placed at the proximal end of guidewire to achieve force feedback, but due to the non-rigidity of the guidewire and the influence of mechanism friction, the proximal force collected has certain errors. In addition, the current VIS robotic systems are also insufficient in functionality, and cannot simultaneously complete the delivery of multiple surgical instruments. To solve the above mechanism design and force feedback challenges, a novel VIS robotic system equipped with force sensing mechanism is developed in this study. In addition, a temporal convolutional network (TCN) for the guidewire distal force prediction and an enhanced interactive force feedback strategy are proposed to improve the safety of the robotic system. Finally, combining the developed robotic system and the enhanced interactive force feedback strategy, a series of performance evaluations and model experiments are carried out. The results of the study demonstrate the effectiveness of the developed robotic system and the feasibility of the enhanced interactive force feedback strategy in improving surgical safety.
Recently, global attention has been paid to climate change. On this account, the market-based carbon pricing scheme is developed to limit greenhouse gas emissions, where a proper grasp of the pricing ...mechanism is crucial for alleviating global warming. Accordingly, we propose a novel method to interpret carbon price dynamics, concurrently deriving the precise prediction and causality. Due to the nonlinearity and nonstationarity of carbon prices, we develop a real-time decomposition approach coupling the multiple ensemble patch transform (MEPT) and the improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN). The MEPT captures the multi-resolution trends of the carbon prices series exactly, and then the ICEEMDAN extracts the fluctuation patterns. Additionally, we collect the numerous potential factors, involving energy sources, energy prices, stock market indices, and economic information. Furthermore, we developed causal temporal convolutional networks (CTCNs) to realize the accurate prediction and the proper causal inference simultaneously. The experimental results on the European Union Allowance (EUA) confirm the effectiveness and necessity of the real-time MEPT-ICEEMDAN decomposition. Moreover, the proposed MEPT-ICEEMDAN-CTCN model exhibits significant superiority in multi-step-ahead and quantile forecast, which realizes the 0.73881%, 1.04461%, and 1.23495% MAPE in one-, five-, and ten-step-ahead forecast respectively and 0.00032 PDQ0.1 and the 0.00285 PDQ0.9 in the quantile forecast. Meanwhile, it reveals the nonlinear Granger causality across the various horizons and quantiles for the first time. It is instructive and inspiring for policymakers, carbon-consumed industries, investors, and researchers.
•The Granger forecast model yields prediction and causality concurrently.•The real-time decomposition method extracts the essential features in practice.•The causal temporal convolutional networks are explainable.•The proposed model achieves the most accurate and stable forecast results.•The results show the differences of causality at various carbon price quantiles.
•STDnet-ST is a novel spatio-temporal ConvNet for small object detection.•STDnet-ST exploits the correlation of promising regions between frames.•An efficient tubelet linking is performed to link ...small objects across video frames.•A novel tubelet suppression algorithm is proposed to avoid unprofitable tubelets.•STDnet-ST outperforms its state-of-the-art counterparts for small target detection.
Object detection through convolutional neural networks is reaching unprecedented levels of precision. However, a detailed analysis of the results shows that the accuracy in the detection of small objects is still far from being satisfactory. A recent trend that will likely improve the overall object detection success is to use the spatial information operating alongside temporal video information. This paper introduces STDnet-ST, an end-to-end spatio-temporal convolutional neural network for small object detection in video. We define small as those objects under 16×16 px, where the features become less distinctive. STDnet-ST is an architecture that detects small objects over time and correlates pairs of the top-ranked regions with the highest likelihood of containing those small objects. This permits to link the small objects across the time as tubelets. Furthermore, we propose a procedure to dismiss unprofitable object links in order to provide high quality tubelets, increasing the accuracy. STDnet-ST is evaluated on the publicly accessible USC-GRAD-STDdb, UAVDT and VisDrone2019-VID video datasets, where it achieves state-of-the-art results for small objects.
The inversion of subsurface reservoir properties is of profound significance to the oil and gas energy development and utilization. The strong heterogeneity and complex pore structure of underground ...reservoirs pose a challenge to efficient oil and gas energy development and utilization across the world, which increases the necessity of developing an efficient reservoir properties inversion method. However, traditional model-driven methods are confronted with the challenges of strong nonlinearity and geological heterogeneity. Moreover, previous studies rarely emphasized the importance of nonlinear feature selection and transfer learning (TL). Aiming to address the research gaps, a reservoir properties inversion method was proposed by combining random forest (RF) feature selection, bidirectional temporal convolutional network (BiTCN), bidirectional gated recurrent units (BiGRU) network, multi-head attention (MHA) mechanism and TL strategy. First, the RF was used to screen the features with significant correlation with the target reservoir properties. Thereafter, combining BiTCN and BiGRU network to leverage their complementary strengths, a parallel dual branch feature learning network was constructed to learn richer geological information from logging data. Meanwhile, MHA was introduced to fuse the output features of the dual network structure. Finally, the fused features were passed through the fully connected module to output the inversion results. TL was used to associate the correlation between reservoir properties and model inversion to improve the inversion performance. The application results with actual field data showed that the proposed method was accurate and robust in reservoir properties inversion. This study can provide a new way for reliable reservoir properties inversion and promote the application of artificial intelligence in data-driven energy science.
•A parallel dual branch network for reservoir properties inversion is newly proposed.•A feature selection combining correlation coefficient and random forest is developed.•Transfer learning contributes significantly to improve the inversion accuracy.•Actual data application results show that the method is feasible and effective.
PDF