The electronic nose (e-nose) is susceptible to sensor drift and instrumental variation, which may result in distribution discrepancy in data collected, hence leading to classification performance ...degradation. It is necessary to apply domain adaptation to solve this problem. A nonlinear subspace projection approach named feature entropy domain adaption (FEDA) is proposed for domain adaptation for e-nose data classification. One important aspect of FEDA is that adversarial training is introduced to minimize the distribution discrepancy between source and target domains. No projection matrix and parameter fine-tuning are needed anymore in comparison with the popular linear subspace projection approaches. In addition, feature norm and conditional entropy are introduced into adversarial training in FEDA to reduce the decision boundary uncertainty and the overlap between classes, respectively. Experimental results show that the FEDA can deal with the distribution discrepancy of e-nose effectively, and can achieve satisfactory classification accuracy on various datasets. Source code can be found at https://github.com/threedteam/DA_FEDA .
Layout guides the position and scale of design elements for desirable aesthetics and effective demonstration. Recently, Generative Adversarial Networks (GANs) have proved their capability in ...generating effective layouts. However, current GANs ignore the situation where the amounts and types of the input design elements are given and determined. In this paper, we propose EcGAN, an element-conditioned GAN for graphic layout generation conditioned on specified design elements (design elements’ amount and types). We represent each element by a bounding box and propose three components: element mask, element condition loss and two-step discriminators, to solve the bounding box modelling problem for element-conditioned layout generation. Experiments reveal that EcGAN outperforms existing methods quantitatively and qualitatively. We also perform detailed ablation studies to highlight the effect of each component and a user study to further validate our model. Finally, we demonstrate two of EcGAN’s applications for practical design scenarios.
Automatic generation of artistic glyph images is a challenging task that attracts many research interests. Previous methods either are specifically designed for shape synthesis or focus on texture ...transfer. In this paper, we propose a novel model, AGIS-Net, to transfer both shape and texture styles in one-stage with only a few stylized samples. To achieve this goal, we first disentangle the representations for content and style by using two encoders, ensuring the multi-content and multi-style generation. Then we utilize two collaboratively working decoders to generate the glyph shape image and its texture image simultaneously. In addition, we introduce a local texture refinement loss to further improve the quality of the synthesized textures. In this manner, our one-stage model is much more efficient and effective than other multi-stage stacked methods. We also propose a large-scale dataset with Chinese glyph images in various shape and texture styles, rendered from 35 professional-designed artistic fonts with 7,326 characters and 2,460 synthetic artistic fonts with 639 characters, to validate the effectiveness and extendability of our method. Extensive experiments on both English and Chinese artistic glyph image datasets demonstrate the superiority of our model in generating high-quality stylized glyph images against other state-of-the-art methods.
We introduce a generative model for 3D man-made shapes. The presented method takes a global-to-local (G2L) approach. An adversarial network (GAN) is built first to construct the overall structure of ...the shape, segmented and labeled into parts. A novel conditional auto-encoder (AE) is then augmented to act as a part-level refiner. The GAN, associated with additional local discriminators and quality losses, synthesizes a voxel-based model, and assigns the voxels with part labels that are represented in separate channels. The AE is trained to amend the initial synthesis of the parts, yielding more plausible part geometries. We also introduce new means to measure and evaluate the performance of an adversarial generative model. We demonstrate that our global-to-local generative model produces significantly better results than a plain three-dimensional GAN, in terms of both their shape variety and the distribution with respect to the training data.
Deep learning is a rapidly developing approach in the field of infrared and visible image fusion. In this context, the use of dense blocks in deep networks significantly improves the utilization of ...shallow information, and the combination of the Generative Adversarial Network (GAN) also improves the fusion performance of two source images. We propose a new method based on dense blocks and GANs , and we directly insert the input image-visible light image in each layer of the entire network. We use structural similarity and gradient loss functions that are more consistent with perception instead of mean square error loss. After the adversarial training between the generator and the discriminator, we show that a trained end-to-end fusion network – the generator network – is finally obtained. Our experiments show that the fused images obtained by our approach achieve good score based on multiple evaluation indicators. Further, our fused images have better visual effects in multiple sets of contrasts, which are more satisfying to human visual perception.
•Dense connection and skip connection of visible images operations are added to the generator network.•A reasonable loss function is designed to replace the mean square error loss of the image.•The discriminator is used to enhance the perception of our images.•The proposed method achieves better performance compare with existing methods.
Underwater image enhancement (UIE) is an essential task for intelligent environment perception in underwater remote visual sensing scenarios. However, the computing power of mobile platforms limits ...the usage of larger scale models. In this article, we propose a lightweight encoder-decoder architecture UIE network (UIENet) to enhance underwater images from visual sensors. We also involve the architecture into a generative adversarial network (UIEGAN) model against a supervised discriminator to further perfect its corrective capabilities for the photorealistic images with more global appearance and local details. The multiresolution counterparts are embedded into the generator to diversify the feature representation of the original inputs. Further, UIEGAN guides the spatial attention module (SAM) and the channel attention module (CAM) to jointly enhance the global-local connection of the image. We evaluate the proposed method on benchmark datasets of UIEB and UFO-120 and report better performance than the state-of-the-art (SOTA) schemes, exceeding 15.43% and 12.85% on peak signal-to-noise ratio (PSNR) than the baselines of these datasets. Besides, by testing on the UIEB challenge, URPC and SQUID datasets without any reference images, our scheme outperforms the other methods on evaluation metrics to validate its generalization performance, and meanwhile uses a series of ablation study demonstrates the effectiveness of the functional modules.
The quick spread of coronavirus disease (COVID-19) has resulted in a global pandemic and more than fifteen million confirmed cases. To battle this spread, clinical imaging techniques, for example, ...computed tomography (CT), can be utilized for diagnosis. Automatic identification software tools are essential for helping to screen COVID-19 using CT images. However, there are few datasets available, making it difficult to train deep learning (DL) networks. To address this issue, a generative adversarial network (GAN) is proposed in this work to generate more CT images. The Whale Optimization Algorithm (WOA) is used to optimize the hyperparameters of GAN's generator. The proposed method is tested and validated with different classification and meta-heuristics algorithms using the SARS-CoV-2 CT-Scan dataset, consisting of COVID-19 and non-COVID-19 images. The performance metrics of the proposed optimized model, including accuracy (99.22%), sensitivity (99.78%), specificity (97.78%), F1-score (98.79%), positive predictive value (97.82%), and negative predictive value (99.77%), as well as its confusion matrix and receiver operating characteristic (ROC) curves, indicate that it performs better than state-of-the-art methods. This proposed model will help in the automatic screening of COVID-19 patients and decrease the burden on medicinal services frameworks.
Polarization imaging has become a promising way for clear underwater vision, which depends on the difference of polarization characteristics between backscattered light and target signal. In this ...paper, to achieve clear underwater color polarization imaging, we propose a learning-based method that uses multi-polarization fusion adversarial generative networks to learn the relationship between polarization information and object radiance. The proposed method is especially designed for multiple polarimetric images, which can effectively extract the polarization correlations of images with different polarization states. Moreover, to train the proposed network, we build, for the first time to our knowledge, a color polarization image dataset from natural underwater environments through passive polarization imaging. The experimental results in laboratory and natural underwater environments show that it is feasible to introduce polarization information into learning-based image recovery, and deep learning technology is conducive to the extraction of polarization information. Comparing with other methods, the proposed method can effectively remove the backscattered light and recover the object radiance.
•We exploit TCNs to efficiently model the long-term temporal dependencies of human motion sequences.•We incorporate SN into the model to achieve reproducibility.•Two discriminators are introduced to ...ensure the better performance.•On the 3 human action benchmarks, our model outperforms the state-of-the-art methods.
Human motion prediction from its historical poses is an essential task in computer vision; it is successfully applied for human-machine interaction and intelligent driving. Recently, significant progress has been made with variants of RNNs or LSTMs. Despite alleviating the vanishing gradient problem, the chain RNN often leads to deformities and convergence to the mean pose because of its low ability to capture long-term dependencies. To address these problems, in this paper, we propose a temporal convolutional generative adversarial network (TCGAN) to forecast high-fidelity future poses. The TCGAN uses hierarchical temporal convolution to model the long-term patterns of human motion effectively. In contrast to RNNs, the hierarchical convolution structure has recently proved to be a more efficient method for sequence-to-sequence learning in computational complexity, the number of model parameters, and parallelism. Besides, instead of traditional GANs, spectral normalization (SN) is embedded in the model to alleviate mode collapse. Compared with typical recurrent methods, the proposed model is feedforward and can produce the future poses in real-time. Extensive experiments on various human activity analysis benchmarks (i.e., H3.6M, CMU, and 3DPW MoCap) demonstrate that the model consistently outperforms the state-of-the-art methods in terms of accuracy and visualization for short-term and long-term predictions.
•Confirmed statistics-conforming property of GANs for modeling dynamical systems.•Highlighted the lack of robustness of GANs and need of explicit physical constraints.•Improved training robustness of ...GANs by explicitly enforcing statistical constraints.•Demonstrated merits of statistics-informed GANs on modeling Rayleigh-Bénard convection.
Simulating complex physical systems often involves solving partial differential equations (PDEs) with some closures due to the presence of multi-scale physics that cannot be fully resolved. Although the advancement of high performance computing has made resolving small-scale physics possible, such simulations are still very expensive. Therefore, reliable and accurate closure models for the unresolved physics remains an important requirement for many computational physics problems, e.g., turbulence simulation. Recently, several researchers have adopted generative adversarial networks (GANs), a novel paradigm of training machine learning models, to generate solutions of PDEs-governed complex systems without having to numerically solve these PDEs. However, GANs are known to be difficult in training and likely to converge to local minima, where the generated samples do not capture the true statistics of the training data. In this work, we present a statistical constrained generative adversarial network by enforcing constraints of covariance from the training data, which results in an improved machine-learning-based emulator to capture the statistics of the training data generated by solving fully resolved PDEs. We show that such a statistical regularization leads to better performance compared to standard GANs, measured by (1) the constrained model's ability to more faithfully emulate certain physical properties of the system and (2) the significantly reduced (by up to 80%) training time to reach the solution. We exemplify this approach on the Rayleigh-Bénard convection, a turbulent flow system that is an idealized model of the Earth's atmosphere. With the growth of high-fidelity simulation databases of physical systems, this work suggests great potential for being an alternative to the explicit modeling of closures or parameterizations for unresolved physics, which are known to be a major source of uncertainty in simulating multi-scale physical systems, e.g., turbulence or Earth's climate.
PDF
