Health indicator (HI) affects the accuracy and reliability of the remaining useful life (RUL) prediction model. The hidden variables of variational autoencoder (VAE) can represent the HI values for a ...life-cycle dataset with obvious degradation trend. However, for an irregular dataset of a rotary machine, it is still a great challenge to construct the HI that can effectively represent the machinery degradation tendency. Therefore, this article proposes a novel degradation-trend-constrained VAE (DTC-VAE) to construct the HI vector with the distinct degradation trend. First, the multidimensional time-domain and frequency-domain characteristics are calculated via the collected vibration samples. Second, a new degradation-constraint loss term is proposed and introduced into VAE for constructing DTC-VAE. Third, with the multidimensional features and DTC-VAE, various HIs can be generated without supervision. The proposed method is applied to construct the HI vectors of bearing life-cycle datasets and gear fatigue datasets, and then macroscopic-microscopic-attention-based long short term memory (MMALSTM) is used to predict the corresponding RULs with the constructed HIs. Via several contrast experiments, the results prove that the proposed unsupervised HI construction approach is superior to other typical methods, and the obtained HI vectors are more suitable for the RUL prediction.
Deep learning (DL) algorithms have received increased attention in various fields. In the field of geoscience, DL has been shown to be a powerful tool for mining complex, high-level, and non-linear ...geospatial data and for extracting previously unknown patterns related to geological processes. In this study, a deep variational autoencoder (VAE) network was used to extract features related to mineralization; and these features were then integrated as a anomaly map in support of mineral exploration based on geochemical exploration data, which consist of Cu, Pb, Mn, Zn and Fe2O3. Various experiments were conducted to determine the optimal parameters of the VAE. The structure of the VAE, in which the network depth and number of hidden units were 24–12-3-12-24, was built to recognize geochemical anomalies related to Fe polymetallic mineralization in the southwest Fujian Province, China. The geochemical anomalies recognized by the VAE show a close spatial correlation with known Fe polymetallic deposits. Meanwhile, the areas with high probability are located in or around the Yanshanian intrusions and the contact zones of the Carboniferous–Permian formation and Yanshanian intrusions. These results suggest that the anomalous areas identified by the VAE are meaningful for mineral exploration.
•A deep variational autoencoder (VAE) network for multivariate geochemical anomalies recognition is demonstrated.•The reconstruction probability instead of reconstruction error is employed as the anomaly score.•A case study from southwestern Fujian district is conducted.
High labor costs and the requirement for significant domain expertise often result in a lack of anomaly labels in most time series. Consequently, employing unsupervised methods becomes critical for ...practical industrial applications. However, prevailing reconstruction-based anomaly detection algorithms encounter challenges in capturing intricate underlying correlations and temporal dependencies in time series. This study introduces an unsupervised anomaly detection model called Variational AutoeEncoder with Adversarial Training for Multivariate Time Series Anomaly Detection (VAEAT). Its fundamental concept involves adopting a two-phase training strategy to improve anomaly detection precision through adversarial reconstruction of raw data. In the first phase, the model reconstructs raw data to extract its basic features by training two enhanced variational autoencoders (VAEs) that incorporate both the long short-term memory (LSTM) network and the attention mechanism in their common encoder. In the second phase, the model refines reconstructed data to optimize the reconstruction quality. In this manner, this two-phase VAE model effectively captures intricate underlying correlation and temporal dependencies. A large number of experiments are conducted to evaluate the performance on five publicly available datasets, and experimental results illustrate that VAEAT exhibits robust performance and effective anomaly detection capabilities. The source code of the proposed VAEAT can be available at https://github.com/Du-Team/VAEAT.
•Developed deep learning methods to forecast the COVID19 spread.•Five deep learning models have been compared for COVID-19 forecasting.•Time-series COVID19 data from Italy, Spain, France, China, the ...USA, and Australia are used.•Results demonstrate the potential of deep learning models to forecast COVID19 data.•Results show the superior performance of the Variational AutoEncoder model.
The novel coronavirus (COVID-19) has significantly spread over the world and comes up with new challenges to the research community. Although governments imposing numerous containment and social distancing measures, the need for the healthcare systems has dramatically increased and the effective management of infected patients becomes a challenging problem for hospitals. Thus, accurate short-term forecasting of the number of new contaminated and recovered cases is crucial for optimizing the available resources and arresting or slowing down the progression of such diseases. Recently, deep learning models demonstrated important improvements when handling time-series data in different applications. This paper presents a comparative study of five deep learning methods to forecast the number of new cases and recovered cases. Specifically, simple Recurrent Neural Network (RNN), Long short-term memory (LSTM), Bidirectional LSTM (BiLSTM), Gated recurrent units (GRUs) and Variational AutoEncoder (VAE) algorithms have been applied for global forecasting of COVID-19 cases based on a small volume of data. This study is based on daily confirmed and recovered cases collected from six countries namely Italy, Spain, France, China, USA, and Australia. Results demonstrate the promising potential of the deep learning model in forecasting COVID-19 cases and highlight the superior performance of the VAE compared to the other algorithms.
To realize the anomaly detection for industrial multi-sensor data, we develop a novel multi-scale convolutional recurrent variational autoencoder (MSCRVAE) model. It is a hybrid of convolutional ...autoencoder and convolutional long short-term memory with variational autoencoder (ConvLSTM-VAE). The ConvLSTM-VAE part helps the MSCRVAE model not only capture the spatial and temporal dependence of data but also learn robust high-level representations by estimating the distributions of latent variables. Moreover, a typical loss function that combines the characteristics of both autoencoder and variational autoencoder is designed for the MSCRVAE model. The experimental results on three public data illustrate the superiority of the MSCRVAE model in anomaly detection with the best average F1-score up to 0.90. The interquartile range of the boxplots on the public data also proves the robustness of the MSCRVAE model. We also illustrate the role of the two components in the loss function by exploring their changes during training. What is more, on the private data, the MSCRVAE model also performs well and the residual matrices provide reasonable interpretations to the anomaly detection results.
•We propose the MSCRVAE model to detect anomaly for industrial multi-sensor data.•It is a reconstruction-based model with the hybrid of ConvAE and ConvLSTM-VAE.•A typical loss function is designed for MSCRVAE and its efficacy is illustrated.•MSCRVAE performs very well on three public data and the private industrial data.•MSCRVAE shows great interpretability in anomaly detection for multi-sensor data.
In recent years, clustering methods based on deep generative models have received great attention in various unsupervised applications, due to their capabilities for learning promising latent ...embeddings from original data. This article proposes a novel clustering method based on variational autoencoder (VAE) with spherical latent embeddings. The merits of our clustering method can be summarized as follows. First, instead of considering the Gaussian mixture model (GMM) as the prior over latent space as in a variety of existing VAE-based deep clustering methods, the von Mises-Fisher mixture model prior is deployed in our method, leading to spherical latent embeddings that can explicitly control the balance between the capacity of decoder and the utilization of latent embedding in a principled way. Second, a dual VAE structure is leveraged to impose the reconstruction constraint for the latent embedding and its corresponding noise counterpart, which embeds the input data into a hyperspherical latent space for clustering. Third, an augmented loss function is proposed to enhance the robustness of our model, which results in a self-supervised manner through the mutual guidance between the original data and the augmented ones. The effectiveness of the proposed deep generative clustering method is validated through comparisons with state-of-the-art deep clustering methods on benchmark datasets. The source code of the proposed model is available at https://github.com/fwt-team/DSVAE .
The advances in thermal metamaterials and their applications have revolutionized how we can manipulate thermal transport behavior. The challenging inverse design problems of utilizing thermal ...metamaterial-based structures to achieve desired thermal transport behavior are increasingly being tackled by data-driven, machine learning-based approaches. The explosive progress in generative AI is permeating the field of material design by offering new perspectives to address the inverse design problems. In this paper, we propose a simple yet effective method of training a generative conditional variational autoencoder to find the design parameters for a thermal metamaterial-based system with a periodic interparticle arrangement to achieve thermal transparency, which is one of the most desirable and interesting thermal transport behaviors. Our work attests to the predictive power of a generative model with a relatively small number of parameters for the purpose of tackling inverse design problems to achieve thermal transport behavior manipulation.
•Thermal metamaterials empower us to manipulate thermal transport behavior.•Generative AI models are powerful tools to address the inverse design problems.•Conditional variational encoder efficiently addresses the inverse design problems.•The obtained metamaterial-based structure achieves thermal transparency.
Deep generative adversarial networks (GANs) are the emerging technology in drug discovery and biomarker development. In our recent work, we demonstrated a proof-of-concept of implementing deep ...generative adversarial autoencoder (AAE) to identify new molecular fingerprints with predefined anticancer properties. Another popular generative model is the variational autoencoder (VAE), which is based on deep neural architectures. In this work, we developed an advanced AAE model for molecular feature extraction problems, and demonstrated its advantages compared to VAE in terms of (a) adjustability in generating molecular fingerprints; (b) capacity of processing very large molecular data sets; and (c) efficiency in unsupervised pretraining for regression model. Our results suggest that the proposed AAE model significantly enhances the capacity and efficiency of development of the new molecules with specific anticancer properties using the deep generative models.
PDF
