Variable rate is a requirement for flexible and adaptable image and video compression. However, deep image compression methods (DIC) are optimized for a single fixed rate-distortion (R-D) tradeoff. ...While this can be addressed by training multiple models for different tradeoffs, the memory requirements increase proportionally to the number of models. Scaling the bottleneck representation of a shared autoencoder can provide variable rate compression with a single shared autoencoder. However, the R-D performance using this simple mechanism degrades in low bitrates, and also shrinks the effective range of bitrates. To address these limitations, we formulate the problem of variable R-D optimization for DIC, and propose modulated autoencoders (MAEs), where the representations of a shared autoencoder are adapted to the specific R-D tradeoff via a modulation network. Jointly training this modulated autoencoder and the modulation network provides an effective way to navigate the R-D operational curve. Our experiments show that the proposed method can achieve almost the same R-D performance of independent models with significantly fewer parameters.
The last decades have witnessed a vast amount of interest and research in feature representation learning from multiple disciplines, such as biology and bioinformatics. Among all the real-world ...application scenarios, feature extraction from knowledge graph (KG) for personalized recommendation has achieved substantial performance for addressing the problem of information overload. However, the rating matrix of recommendations is usually sparse, which may result in significant performance degradation. The crucial problem is how to extract and extend features from additional side information. To address these issues, we propose a novel feature representation learning method for the recommendation in this paper that extends item features with knowledge graph via triple-autoencoder. More specifically, the comment information between users and items is first encoded as sentiment classification. These features are then applied as the input to the autoencoder for generating the auxiliary information of items. Second, the item-based rating, the side information, and the generated comment representations are incorporated into the semi-autoencoder for reconstructed output. The low-dimensional representations of this extended information are learned with the semi-autoencoder. Finally, the reconstructed output generated by the semi-autoencoder is input into a third autoencoder. A serial connection between the semi-autoencoder and the autoencoder is designed here to learn more abstract and higher-level feature representations for personalized recommendation. Extensive experiments conducted on several real-world datasets validate the effectiveness of the proposed method compared to several state-of-the-art models.
Effective fault diagnosis has long been a research topic in the prognosis and health management of rotary machinery engineered systems due to the benefits such as safety guarantees, reliability ...improvements, and economical efficiency. This paper investigates an effective and reliable deep learning method known as stacked denoising autoencoder (SDA), which is shown to be suitable for certain health state identifications for signals containing ambient noise and working condition fluctuations. SDA has become a popular approach to achieve the promised advantages of deep architecture-based robust feature representations. In this paper, the SDA-based fault diagnosis method contains three successive steps: health states are first divided into training and testing groups for the SDA model, a deep hierarchical structure is then established with a transmitting rule of greedy training, layer by layer, where sparsity representation and data destruction are applied to obtain high-order characteristics with better robustness in the iteration learning. Validation data are finally employed to confirm the fault diagnosis results of the SDA, where existing health state identification methods are used for comparison. Rotating machinery datasets are employed to demonstrate the effectiveness of the proposed method.
•Deep neural network is developed for fault diagnosis of typical dynamic systems.•Better robustness is achieved under various working conditions and ambient noise.•The method helps salient fault characteristic mining and intelligent diagnosis.•Validity of the SDA is verified via comparative experiments.
Spectral unmixing (SU), which refers to extracting basic features (i.e., endmembers) at the subpixel level and calculating the corresponding proportion (i.e., abundances), has become a major ...preprocessing technique for the hyperspectral image analysis. Since the unmixing procedure can be explained as finding a set of low-dimensional representations that reconstruct the data with their corresponding bases, autoencoders (AEs) have been effectively designed to address unsupervised SU problems. However, their ability to exploit the prior properties remains limited, and noise and initialization conditions will greatly affect the performance of unmixing. In this article, we propose a novel technique network for unsupervised unmixing which is based on the adversarial AE, termed as adversarial autoencoder network (AAENet), to address the above problems. First, the image to be unmixed is assumed to be partitioned into homogeneous regions. Then, considering the spatial correlation between local pixels, the pixels in the same region are assumed to share the same statistical properties (means and covariances) and abundance can be modeled to follow an appropriate prior distribution. Then the adversarial training procedure is adapted to transfer the spatial information into the network. By matching the aggregated posterior of the abundance with a certain prior distribution to correct the weight of unmixing, the proposed AAENet exhibits a more accurate and interpretable unmixing performance. Compared with the traditional AE method, our approach can greatly enhance the performance and robustness of the model by using the adversarial procedure and adding the abundance prior to the framework. The experiments on both the simulated and real hyperspectral data demonstrate that the proposed algorithm can outperform the other state-of-the-art methods.
Natural convection in porous media is a highly nonlinear multiphysical problem relevant to many engineering applications (e.g., the process of CO2 sequestration). Here, we extend and present a ...non-intrusive reduced order model of natural convection in porous media employing deep convolutional autoencoders for the compression and reconstruction and either radial basis function (RBF) interpolation or artificial neural networks (ANNs) for mapping parameters of partial differential equations (PDEs) on the corresponding nonlinear manifolds. To benchmark our approach, we also describe linear compression and reconstruction processes relying on proper orthogonal decomposition (POD) and ANNs. We present comprehensive comparisons among different models through three benchmark problems. The reduced order models, linear and nonlinear approaches, are much faster than the finite element model, obtaining a maximum speed-up of 7×106 because our framework is not bound by the Courant–Friedrichs–Lewy condition; hence, it could deliver quantities of interest at any given time contrary to the finite element model. Our model’s accuracy still lies within a relative error of 7% in the worst-case scenario. We illustrate that, in specific settings, the nonlinear approach outperforms its linear counterpart and vice versa. We hypothesize that a visual comparison between principal component analysis (PCA) and t-Distributed Stochastic Neighbor Embedding (t-SNE) could indicate which method will perform better prior to employing any specific compression strategy.
•A data-driven framework with deep convolutional autoencoders to approximate natural convection in porous media is developed.•The framework can handle data with adaptive time-stepping and deliver the solution at any time between the training snapshots.•Visualization of latent space can be used to select an appropriate method between linear and nonlinear approaches.
Extreme learning machine (ELM) is an efficient learning algorithm of training single layer feed-forward neural networks (SLFNs). With the development of unsupervised learning in recent years, ...integrating ELM with autoencoder has become a new perspective for extracting feature using unlabeled data. In this paper, we propose a new variant of extreme learning machine autoencoder (ELM-AE) called generalized extreme learning machine autoencoder (GELM-AE) which adds the manifold regularization to the objective of ELM-AE. Some experiments carried out on real-world data sets show that GELM-AE outperforms some state-of-the-art unsupervised learning algorithms, including k-means, laplacian embedding (LE), spectral clustering (SC) and ELM-AE. Furthermore, we also propose a new deep neural network called multilayer generalized extreme learning machine autoencoder (ML-GELM) by stacking several GELM-AE to detect more abstract representations. The experiments results show that ML-GELM outperforms ELM and many other deep models, such as multilayer ELM autoencoder (ML-ELM), deep belief network (DBN) and stacked autoencoder (SAE). Due to the utilization of ELM, ML-GELM is also faster than DBN and SAE.
Multi-spatial-resolution change detection is a newly proposed issue and it is of great significance in remote sensing, environmental and land use monitoring, etc. Though multi-spatial-resolution ...image-pair are two kinds of representations of the same reality, they are often incommensurable superficially due to their different modalities and properties. In this paper, we present a novel multi-spatial-resolution change detection framework, which incorporates deep-architecture-based unsupervised feature learning and mapping-based feature change analysis. Firstly, we transform multi-resolution image-pair into the same pixel-resolution through co-registration, followed by details recovery, which is designed to remedy the spatial details lost in the registration. Secondly, the denoising autoencoder is stacked to learn local and high-level representation/feature from the local neighborhood of the given pixel, in an unsupervised fashion. Thirdly, motivated by the fact that multi-resolution image-pair share the same reality in the unchanged regions, we try to explore the inner relationships between them by building a mapping neural network. And it can be used to learn a mapping function based on the most-unlikely-changed feature-pairs, which are selected from all the feature-pairs via a coarse initial change map generated in advance. The learned mapping function can bridge the different representations and highlight changes. Finally, we can build a robust and contractive change map through feature similarity analysis, and the change detection result is obtained through the segmentation of the final change map. Experiments are carried out on four real datasets, and the results confirmed the effectiveness and superiority of the proposed method.
In recent years, autoencoders and neural network technologies have been widely studied and applied to abnormal data detection problems of industrial data such as bearing vibration, but there are ...still problems such as large training data, network parameter initialization, low training efficiency, poor detection effect and so on. To solve such problems, this paper presents an anomaly data detection method combining Mahalanobis distance and autoencoder network. There is a certain correlation between bearing vibration data characteristics, so the Mahalanobis distance of the data is used to quickly detect some abnormal data, which reduces the amount of training data for the self-encoding network. In this research, the autoencoder and the classifier are combined to construct the autoencoder network, which solves the problem of network parameter initialization and significantly improves the training efficiency. The Mahalanobis distance of the data is added to the data features, which improves the anomaly detection
Since the proposal of a fast learning algorithm for deep belief networks in 2006, the deep learning techniques have drawn ever-increasing research interests because of their inherent capability of ...overcoming the drawback of traditional algorithms dependent on hand-designed features. Deep learning approaches have also been found to be suitable for big data analysis with successful applications to computer vision, pattern recognition, speech recognition, natural language processing, and recommendation systems. In this paper, we discuss some widely-used deep learning architectures and their practical applications. An up-to-date overview is provided on four deep learning architectures, namely, autoencoder, convolutional neural network, deep belief network, and restricted Boltzmann machine. Different types of deep neural networks are surveyed and recent progresses are summarized. Applications of deep learning techniques on some selected areas (speech recognition, pattern recognition and computer vision) are highlighted. A list of future research topics are finally given with clear justifications.
Intelligent fault diagnosis based on deep learning (DL) has been widely used in various engineering practices. However, when confronting massive unlabeled industrial data, traditional data-driven ...intelligent fault diagnosis approaches cannot fully mine the correlation geometric structure information between data, and therefore cannot obtain good fault diagnosis results. To overcome this difficulty, an efficient fault diagnosis method based on deep hypergraph autoencoder embedding (DHAEE) is presented in this study. First, unlabeled vibration signals are converted into hypergraphs by applying the designed hypergraph construction method. Second, a hypergraph convolutional extreme learning machine autoencoder (HCELM-AE) is designed, which can mine the higher-order structural information and subspace structural information of the original unlabeled data by designing hypergraph convolutional and self-representation layers. Furthermore, by stacking multiple HCELM-AE modules in a DL framework, the DHAEE and its corresponding fault diagnosis method is constructed, which not only has the advantage of high computational efficiency of ELM-AE, but also has strong representational learning ability of DL methods. Finally, the effectiveness of the DHAEE based fault diagnosis method is verified by rolling bearing fault data and rotor fault data. Experimental results show that the presented fault diagnosis method has higher accuracy and lower computational complexity than other comparison methods, thus proving that DHAEE is an efficient intelligent massive unlabeled data processing approach for rotating machinery fault diagnosis.
PDF
