•Proposes a novel resampling method based on a deep generative model to deal with the imbalanced regression data sets.•Improves the effect of undersampling and reduces the number of normal samples ...without losing informative samples by the proposed undersampling method.•Improves the quality of new rare regression samples, can better capture and learn information from rare samples by introducing the boosting mechanism and deep generative model.
Resampling is the most commonly used method for dealing with imbalanced data, in addition to modifying the algorithm mechanism, it can, for example, generate new minority samples or reduce majority samples to adjust the data distribution. However, to date, related research has predominantly focused on solving the classification problem, while the issue of imbalanced regression data has rarely been discussed. In real-world applications, predicting regression data is a common and valuable issue in decision making, especially in regard to those rare samples with extremely high or low values, such as those encountered in the fields of signal processing, finance, or meteorology. This study therefore divided its regression data into rare samples and normal samples, with self-defined relevance functions and, in addition, proposed a boosting resampling method based on a conditional variational autoencoder. The experimental results showed that when using the proposed resampling method was employed, the prediction performance of the whole testing data set was slightly increased, while the performance for the rare samples was significantly improved.
Semantic communications have emerged as a new paradigm for improving communication efficiency by transmitting the semantic information of a source message that is most relevant to a desired task at ...the receiver. Most existing approaches typically utilize neural networks (NNs) to design end-to-end semantic communication systems, where NN-based semantic encoders output continuously distributed signals to be sent directly to the channel in an analog fashion. In this work, we propose a joint coding-modulation (JCM) framework for digital semantic communications by using variational autoencoder (VAE). Our approach learns the transition probability from source data to discrete constellation symbols, thereby avoiding the non-differentiability problem of digital modulation. Meanwhile, by jointly designing the coding and modulation process together, we can match the obtained modulation strategy with the operating channel condition. We also derive a matching loss function with information-theoretic meaning for end-to-end training. Experiments on image semantic communication validate the superiority of our proposed JCM framework over the state-of-the-art quantization-based digital semantic coding-modulation methods across a wide range of channel conditions, transmission rates, and modulation orders. Furthermore, its performance gap to analog semantic communication reduces as the modulation order increases while enjoying the hardware implementation convenience.
Unsupervised outlier detection is a vital task and has high impact on a wide variety of applications domains, such as image analysis and video surveillance. It also gains long-standing attentions and ...has been extensively studied in multiple research areas. Detecting and taking action on outliers as quickly as possible are imperative in order to protect network and related stakeholders or to maintain the reliability of critical systems. However, outlier detection is difficult due to the one class nature and challenges in feature construction. Sequential anomaly detection is even harder with more challenges from temporal correlation in data, as well as the presence of noise and high dimensionality. In this paper, we introduce a novel deep structured framework to solve the challenging sequential outlier detection problem. We use autoencoder models to capture the intrinsic difference between outliers and normal instances and integrate the models to recurrent neural networks that allow the learning to make use of previous context as well as make the learners more robust to warp along the time axis. Furthermore, we propose to use a layerwise training procedure, which significantly simplifies the training procedure and hence helps achieve efficient and scalable training. In addition, we investigate a fine-tuning step to update all parameters set by incorporating the temporal correlation in the sequence. We further apply our proposed models to conduct systematic experiments on five real-world benchmark data sets. Experimental results demonstrate the effectiveness of our model, compared with other state-of-the-art approaches.
A physical unclonable function (PUF) is broadly investigated as a secret key generator for internet-of-things (IoT) devices because of its uniqueness and randomness. Security vulnerability may occur ...in conventional PUF-based schemes if an attacker eavesdrops on the challenge and response pair (CRP) associated with PUF. To mitigate the impact of such key compromise, this study proposes a novel dynamic PUF key generation scheme, where the sensing data-based dynamic features are integrated to a static PUF-based key. The compressive autoencoders (AEs) typically employed for compressing the time-series data can be customized to extract data-based features. The dynamic features are quantized to generate the data-based key, and then combined with the PUF-based key for the dynamic key generation. An attacker finds it difficult to extract the PUF-based key from the synthesized ones; hence, CRPs cannot be estimated, even if the attacker can obtain the dynamic key through eavesdropping. The event signal decomposition algorithm, which disjoins the dominant event signal in the dataset, is proposed to enhance the data-based key diversity. For numerical evaluation, the static random access memory-PUF and the public energy dataset are used for the dynamic key generation. Numerical results show that the AE model can generate a data-based key that prohibits key leakage to the attacker while improving the reconstruction performance. The dynamic key is evaluated for diverse attack models (e.g., replay and modeling attacks), and the attacker cannot estimate the CRP tables.
Accurate and timely traffic flow forecasting is critical for the successful deployment of intelligent transportation systems. However, it is quite challenging to develop an efficient and robust ...forecasting model due to the inherent randomness and large variations of traffic flow. Recently, the stacked autoencoder has been proven promising for traffic flow forecasting but still exists some drawbacks in certain conditions. In this paper, a training samples replication strategy is introduced to train a series of stacked autoencoders and an adaptive boosting scheme is proposed to ensemble the trained stacked autoencoders to improve the accuracy of traffic flow forecasting. Furthermore, sufficient experiments have been conducted to demonstrate the superior performance of the proposal.
We demonstrate a new deep learning autoencoder network, trained by a nonnegativity constraint algorithm (nonnegativity-constrained autoencoder), that learns features that show part-based ...representation of data. The learning algorithm is based on constraining negative weights. The performance of the algorithm is assessed based on decomposing data into parts and its prediction performance is tested on three standard image data sets and one text data set. The results indicate that the nonnegativity constraint forces the autoencoder to learn features that amount to a part-based representation of data, while improving sparsity and reconstruction quality in comparison with the traditional sparse autoencoder and nonnegative matrix factorization. It is also shown that this newly acquired representation improves the prediction performance of a deep neural network.
In Anbetracht der erheblichen Umweltauswirkungen des Bauwesens wird die Analyse und v. a. Optimierung der Nachhaltigkeit von Strukturen unter Beibehaltung des etablierten Zuverlässigkeitsniveaus ...immer wichtiger. Im Hochbausektor existieren erste Werkzeuge zur Lebenszyklusanalyse, diese sind jedoch nicht direkt übertragbar auf Brückentragwerke. Dieser Beitrag fasst die wesentlichen Ansätze und Ergebnisse von aktuellen Forschungsprojekten der Autoren an der ETH zusammen und erläutert insbesondere einen neuen Deep‐Learning‐basierten Ansatz zur Erkundung und Modellierung des Entwurfsraums parametrischer Brückenmodelle und deren Leistungsbewertungen und veranschaulicht die Anwendung für eine Mehrzieloptimierung von Stahlbetonrahmenbrücken. Zunächst werden Daten unter Verwendung eines parametrischen Brückenmodells sowie der Ankoppelung von Analysesoftware synthetisch generiert und anschließend bedingte variationelle Autoencoder (CVAE) als Metamodell trainiert. Der CVAE dient im Rahmen des konzeptionellen Brückenentwurfs als effizienter Co‐Pilot sowohl für die Vorwärts‐ als auch Rückwärtsanalyse. Die mit dem CVAE durchgeführte Sensitivitätsanalyse zeigt Beziehungen zwischen Entwurfsparametern und/oder Leistungskenngrößen sowie Optimierungspotenziale auf. Das hier vorgestellte integrierte Framework besitzt das Potenzial zur Realisierung einer effizienten Brückenplanung unter insbesondere den Kriterien der Nachhaltigkeit und Tragsicherheit und kann problemlos auf andere parametrische Fragestellungen erweitert werden.
Translation
Parametric modeling and generative deep learning for bridge design
Given the significant environmental impact of the construction industry, the analysis and, above all, optimization of the sustainability of structures while maintaining established levels of reliability are becoming increasingly important. While there are initial tools for life cycle analysis in the building sector, these are not directly transferable to bridge structures. This paper introduces a deep learning‐based approach to explore and model the design space of parametric bridge models and their performance evaluations, illustrating its application for multi‐objective optimization of reinforced concrete frame bridges. Initially, data is synthetically generated using a parametric bridge model and coupling analysis software, followed by training a conditional variational autoencoder (CVAE) as a metamodel. In the context of conceptual bridge design, the CVAE serves as an efficient co‐pilot for both forward and inverse analysis. The sensitivity analysis performed with the CVAE reveals relationships between design parameters and/or performance metrics, highlighting optimization potentials. The integrated framework presented here has the potential to realize efficient bridge design, particularly focusing on sustainability and structural safety criteria, and can be easily extended to other parametric inquiries.
Recently, deep learning has attracted increasing attention for soft sensor applications in industrial processes. Hierarchical features can be learned from massive process data by deep learning, which ...is the key step for quality variable prediction. However, few deep learning algorithms consider the neighborhood structure of data samples for feature extraction in industrial processes. In this paper, a novel stacked neighborhood preserving autoencoder (S-NPAE) is proposed to extract hierarchical neighborhood-preserving features. As for each NPAE, a novel loss function is proposed to reconstruct the input data and preserve the neighborhood structure of the input data simultaneously. By minimizing this loss function, NPAE can efficiently extract the neighborhood-preserved features from its input data. Then, the deep S-NPAE network is constructed by stacking multiple NPAEs in a hierarchical way. Finally, the extracted features can be used for accurate quality prediction in soft sensor modeling. The experimental results on an industrial hydrocracking process demonstrate the effectiveness of the proposed method when compared with other commonly used methods.
•Dynamic features of process and quality data are learned for quality prediction.•Bi-directional RNN is trained by past and future data to prevent over-fitting.•The unlabeled process data are used to ...enhance the quality prediction performance.•Improved loss functions for different data types enables parallel parameter update.
The online quality variables of soft sensors contribute greatly to obtaining immediate process information. The complex correlations between a large number of process variables inherited from the dynamic and nonlinear characteristics of chemical processes put more challenges on constructing soft-sensor models. Past developed steady-state soft sensors are not reliable for dynamic operating systems. Unequal sampling rates for the process and quality data cause missing values of quality data at some time points. This paper proposes a semi-supervised latent dynamic variational autoencoder to learn features between the process and quality data. A prediction network is constructed to generate artificial quality values for model training. Then the process and quality data are compressed into the latent space and the temporal relation is modeled in the clean latent space. The proposed method is compared with the conventional method for quality prediction in a numerical case and an industrial case.
Anomaly detection is a significant task in sensors' signal processing since interpreting an abnormal signal can lead to making a high-risk decision in terms of sensors' applications. Deep learning ...algorithms are effective tools for anomaly detection due to their capability to address imbalanced datasets. In this study, we took a semi-supervised learning approach, utilizing normal data for training the deep learning neural networks, in order to address the diverse and unknown features of anomalies. We developed autoencoder-based prediction models to automatically detect anomalous data recorded by three electrochemical aptasensors, with variations in the signals' lengths for particular concentrations, analytes, and bioreceptors. Prediction models employed autoencoder networks and the kernel density estimation (KDE) method for finding the threshold to detect anomalies. Moreover, the autoencoder networks were vanilla, unidirectional long short-term memory (ULSTM), and bidirectional LSTM (BLSTM) autoencoders for the training stage of the prediction models. However, the decision-making was based on the result of these three networks and the integration of vanilla and LSTM networks' results. The accuracy as a performance metric of anomaly prediction models showed that the performance of vanilla and integrated models were comparable, while the LSTM-based autoencoder models showed the least accuracy. Considering the integrated model of ULSTM and vanilla autoencoder, the accuracy for the dataset with the lengthier signals was approximately 80%, while it was 65% and 40% for the other datasets. The lowest accuracy belonged to the dataset with the least normal data in its dataset. These results demonstrate that the proposed vanilla and integrated models can automatically detect abnormal data when there is sufficient normal data for training the models.
PDF
