•A deep learning model has been built to address the prediction of diabetes.•Variational auto encoder was trained for sample data augmentation.•Sparse auto encoder was trained for feature ...augmentation.•Results obtained demonstrate the powerful of the model using convolutional layers•A 92.31% of accuracy was obtained outperforming the state of the art techniques.
Background and objective: Diabetes is a chronic pathology which is affecting more and more people over the years. It gives rise to a large number of deaths each year. Furthermore, many people living with the disease do not realize the seriousness of their health status early enough. Late diagnosis brings about numerous health problems and a large number of deaths each year so the development of methods for the early diagnosis of this pathology is essential.
Methods: In this paper, a pipeline based on deep learning techniques is proposed to predict diabetic people. It includes data augmentation using a variational autoencoder (VAE), feature augmentation using an sparse autoencoder (SAE) and a convolutional neural network for classification. Pima Indians Diabetes Database, which takes into account information on the patients such as the number of pregnancies, glucose or insulin level, blood pressure or age, has been evaluated.
Results: A 92.31% of accuracy was obtained when CNN classifier is trained jointly the SAE for featuring augmentation over a well balanced dataset. This means an increment of 3.17% of accuracy with respect the state-of-the-art.
Conclusions: Using a full deep learning pipeline for data preprocessing and classification has demonstrate to be very promising in the diabetes detection field outperforming the state-of-the-art proposals.
Nowadays, data-driven soft sensors have become mainstream for the key performance indicators prediction, which guarantees the safety and stability of the industrial process. The typical autoencoder ...(AE) has been widely used to extract potential features through unsupervised pretraining and supervised fine-tuning. However, most existing studies fail to consider both the time-varying features of the process and the differences in the contributions of the hidden features to the target variable. Therefore, in this article, a stacked spatial-temporal autoencoder (S 2 TAE) is proposed to enhance the representation learning capability for soft sensor modeling by taking the spatial-temporal correlations into consideration. Specifically, to effectively model the temporal dependence from nearby times, a temporal autoencoder is proposed, in which a memory module is devised and integrated to learn valuable historical information. Moreover, a "feature recalibration" block is developed and embedded into the spatial-temporal autoencoder (STAE) to selectively capture more informative features and suppress the less useful ones in a supervised way. Then, multiple STAEs are stacked to construct the S 2 TAE network to extract more robust high-level features. Finally, the experimental results on two real-world datasets of a sorbent decontamination system (SDS) desulfurization process and a high-low transformer demonstrate that the S 2 TAE-based soft sensor is effective and feasible.
•Developed deep learning methods to forecast the COVID19 spread.•Five deep learning models have been compared for COVID-19 forecasting.•Time-series COVID19 data from Italy, Spain, France, China, the ...USA, and Australia are used.•Results demonstrate the potential of deep learning models to forecast COVID19 data.•Results show the superior performance of the Variational AutoEncoder model.
The novel coronavirus (COVID-19) has significantly spread over the world and comes up with new challenges to the research community. Although governments imposing numerous containment and social distancing measures, the need for the healthcare systems has dramatically increased and the effective management of infected patients becomes a challenging problem for hospitals. Thus, accurate short-term forecasting of the number of new contaminated and recovered cases is crucial for optimizing the available resources and arresting or slowing down the progression of such diseases. Recently, deep learning models demonstrated important improvements when handling time-series data in different applications. This paper presents a comparative study of five deep learning methods to forecast the number of new cases and recovered cases. Specifically, simple Recurrent Neural Network (RNN), Long short-term memory (LSTM), Bidirectional LSTM (BiLSTM), Gated recurrent units (GRUs) and Variational AutoEncoder (VAE) algorithms have been applied for global forecasting of COVID-19 cases based on a small volume of data. This study is based on daily confirmed and recovered cases collected from six countries namely Italy, Spain, France, China, USA, and Australia. Results demonstrate the promising potential of the deep learning model in forecasting COVID-19 cases and highlight the superior performance of the VAE compared to the other algorithms.
Representation learning is a crucial and challenging task within multimodal sentiment analysis. Effective multimodal sentiment representations contain two key aspects: consistency and difference. ...However, the state-of-the-art multimodal sentiment analysis approaches failed to capture the difference and consistency of sentiment information across diverse modalities. To address the multimodal sentiment representation problem, we propose an autoencoder-based self-supervised learning framework. In the pre-training stage, an autoencoder is designed for each modality, leveraging unlabeled data to learn richer sentiment representations for each modality through sample reconstruction and modality consistency detection tasks. In the fine-tuning stage, the pre-trained autoencoder is injected into MulT (AE-MT) and enhance the model's ability to extract deep sentiment information by incorporating a contrastive learning auxiliary task. Our experiments on the popular Chinese sentiment analysis benchmark (CH-SIMS v2.0) and English sentiment analysis benchmark (MOSEI) demonstrate significant gains over baseline models.
Deep clustering has achieved great success as its powerful ability to learn effective representation. Especially, graph network clustering has attracted more and more attention. Considering the great ...success of Graph Autoencoder (GAE) in encoding the graph structure and Deep Autoencoder (DAE) in extracting valuable representations from the data itself, in this paper, we construct an Adversarially regularized Joint Structured Clustering Network(AJSCN) by integrating GAE and DAE. The framework links GAE and DAE together by transferring the representation learned by DAE to the corresponding layer of GAE to alleviate the over-smoothing problem. Furthermore, the latent representation learned by GAE is enforced to match a prior distribution via an adversarial training scheme to avoid the free of any structure of latent space. We design a joint supervision mechanism to improve the clustering performance consisting of self-supervision and mutual supervision. The self-supervision is to learn more compact representations, and mutual-supervision makes different representations more consistent. Experiment results demonstrate the superiority of the proposed model against the state-of-the-art algorithms and achieve significant improvement on six benchmark datasets.
A Comprehensive Survey on Graph Neural Networks Wu, Zonghan; Pan, Shirui; Chen, Fengwen ...
IEEE transaction on neural networks and learning systems,
2021-Jan., 2021-01-00, 2021-1-00, Volume:
32, Issue:
1
Journal Article
Open access
Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data ...in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications, where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on the existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this article, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art GNNs into four categories, namely, recurrent GNNs, convolutional GNNs, graph autoencoders, and spatial-temporal GNNs. We further discuss the applications of GNNs across various domains and summarize the open-source codes, benchmark data sets, and model evaluation of GNNs. Finally, we propose potential research directions in this rapidly growing field.
The Denoising Autoencoder (DAE) enhances the flexibility of data stream method in exploiting unlabeled samples. Nonetheless, the feasibility of DAE for data stream analytic deserves in-depth study ...because it characterizes a fixed network capacity which cannot adapt to rapidly changing environments. Deep evolving denoising autoencoder (DEVDAN), is proposed in this paper. It features an open structure in the generative phase and the discriminative phase where the hidden units can be automatically added and discarded on the fly. The generative phase refines the predictive performance of discriminative model exploiting unlabeled data. Furthermore, DEVDAN is free of the problem-specific threshold and works fully in the single-pass learning fashion. We show that DEVDAN can find competitive network architecture compared with state-of-the-art methods on the classification task using ten prominent datasets simulated under the prequential test-then-train protocol.
Fault detection constitutes a fundamental task for predictive maintenance, requiring mathematical models that can be conveniently provided by data-driven techniques. Autoencoders are a particular ...type of unsupervised Artificial Neural Networks that can be suitable for fault detection applications. Diverse architectures might be used for autoencoders, resulting in different fault detection performances, which are usually compared by means of Fault Detection Rates for a fixed threshold of the False Alarm Rate, limiting the conclusions to particular cases. To improve the comparability, the present work uses the area under the receiver operating characteristic curve to compare autoencoder architectures for a range of false alarm rates using the Tennessee Eastman Process benchmark. Performances obtained for shallow and deep autoencoders were compared with those of the denoising and variational autoencoders for undercomplete and sparse structures. Overall, the results indicate better performances for sparse structures, especially for the variational autoencoder and the deep denoising autoencoder, with area under the curve of 98.35%.
Hyperspectral unmixing is to decompose the mixed pixels into pure spectral signatures (endmembers) and their proportions (abundances). Recently, deep learning-based methods have been applied to ...enhance the representation ability of unmixing models by extracting joint spatial-spectral characteristics of the hyperspectral data. However, most deep learning based-unmixing methods usually conduct global smoothing by convolutions on the whole hyperspectral imagery, which may ignore the variations within the imagery and result in oversmoothing. In this article, we propose a deep network for hyperspectral unmixing based on a new global-local smoothing autoencoder (GLA). GLA is an unsupervised model, which aims at exploring the local homogeneity and the global self-similarity of hyperspectral imagery. The proposed GLA network mainly includes two modules: a Local Continuous conditional random field Smoothing (LCS) module and a global recurrent smoothing (GRS) module. In LCS, we propose a conditional random field-based smoothing strategy to describe the joint spatial-spectral information within a local homogeneity region, which also reduces the risk of abundance maps boundary blurry. In GRS, we follow the self-similarity assumption for hyperspectral imagery and develop a recurrent neural network structure to exploit potential long-distance dependency relationships among pixels. The GLA is compared with several state-of-the-art unmixing methods on both real and synthetic data, and the abundance estimation results indicate that our method is promising. We will publish the code of GLA if this article has the honor to be accepted.
PDF
