Single Image Deraining (SID) is a relatively new and still challenging topic in emerging vision applications, and most of the recently emerged deraining methods use the supervised manner depending on ...the ground-truth (i.e., using paired data). However, in practice it is rather common to encounter unpaired images in real deraining task. In such cases, how to remove the rain streaks in an unsupervised way will be a challenging task due to lack of constraints between images and hence suffering from low-quality restoration results. In this paper, we therefore explore the unsupervised SID issue using unpaired data, and propose a new unsupervised framework termed DerainCycleGAN for single image rain removal and generation, which can fully utilize the constrained transfer learning ability and circulatory structures of CycleGAN. In addition, we design an unsupervised rain attentive detector (UARD) for enhancing the rain information detection by paying attention to both rainy and rain-free images. Besides, we also contribute a new synthetic way of generating the rain streak information, which is different from the previous ones. Specifically, since the generated rain streaks have diverse shapes and directions, existing derianing methods trained on the generated rainy image by this way can perform much better for processing real rainy images. Extensive experimental results on synthetic and real datasets show that our DerainCycleGAN is superior to current unsupervised and semi-supervised methods, and is also highly competitive to the fully-supervised ones.
On Data Augmentation for GAN Training Tran, Ngoc-Trung; Tran, Viet-Hung; Nguyen, Ngoc-Bao ...
IEEE transactions on image processing,
2021, Volume:
30
Journal Article
Peer reviewed
Open access
Recent successes in Generative Adversarial Networks (GAN) have affirmed the importance of using more data in GAN training. Yet it is expensive to collect data in many domains such as medical ...applications. Data Augmentation (DA) has been applied in these applications. In this work, we first argue that the classical DA approach could mislead the generator to learn the distribution of the augmented data, which could be different from that of the original data. We then propose a principled framework, termed Data Augmentation Optimized for GAN (DAG), to enable the use of augmented data in GAN training to improve the learning of the original distribution. We provide theoretical analysis to show that using our proposed DAG aligns with the original GAN in minimizing the Jensen-Shannon (JS) divergence between the original distribution and model distribution. Importantly, the proposed DAG effectively leverages the augmented data to improve the learning of discriminator and generator. We conduct experiments to apply DAG to different GAN models: unconditional GAN, conditional GAN, self-supervised GAN and CycleGAN using datasets of natural images and medical images. The results show that DAG achieves consistent and considerable improvements across these models. Furthermore, when DAG is used in some GAN models, the system establishes state-of-the-art Fréchet Inception Distance (FID) scores. Our code is available ( https://github.com/tntrung/dag-gans ).
The accurate degradation performance assessment of rolling bearings is very important for the reliable operation of mechanical equipment. However, most current research is limited to the full life ...cycle signals of outer ring faults. As a vulnerable part of the rolling bearing, the cage is prone to instantaneous fracture. When the bearing cage fails, its signal amplitude surges in a short time, so this poses a certain challenge compared to the outer ring fault analysis. To solve the problem, an improved CycleGAN model is proposed to generate the full life cycle signals of cage degradation across various measuring points. The dilated convolution is introduced into the generator to further expand the model’s receptive field, which enables the model to capture mutation features of the bearing cage fault signal across multiple scales. And the adaptive learning rate decay strategy is applied in the training process to make the model more focused on fault stage characteristics. Moreover, a deep belief network (DBN) optimized with a grid search algorithm is utilized to fit the root mean square value of the full life cycle signal to characterize the bearing cage degradation rule. The real bearing cage accelerated degradation experiment proves that the proposed model can effectively generate the synthetic signals of different measuring points. Meanwhile, the optimized DBN model can fit the degraded data better than the existing methods.
Compressed sensing magnetic resonance imaging (CS-MRI) has provided theoretical foundations upon which the time-consuming MRI acquisition process can be accelerated. However, it primarily relies on ...iterative numerical solvers, which still hinders their adaptation in time-critical applications. In addition, recent advances in deep neural networks have shown their potential in computer vision and image processing, but their adaptation to MRI reconstruction is still in an early stage. In this paper, we propose a novel deep learning-based generative adversarial model, RefineGAN, for fast and accurate CS-MRI reconstruction. The proposed model is a variant of fullyresidual convolutional autoencoder and generative adversarial networks (GANs), specifically designed for CS-MRI formulation; it employs deeper generator and discriminator networks with cyclic data consistency loss for faithful interpolation in the given under-sampled k-space data. In addition, our solution leverages a chained networkto further enhance the reconstruction quality. RefineGAN is fast and accurate-the reconstruction process is extremely rapid, as low as tens of milliseconds for reconstruction of a 256 × 256 image, because it is one-way deployment on a feed-forward network, and the image quality is superior even for extremely low sampling rate (as low as 10%) due to the data-driven nature of the method. We demonstrate that RefineGAN outperforms the state-of-the-art CS-MRI methods by a large margin in terms of both running time and image quality via evaluation using several open-source MRI databases.
Throughout the course of delivering a radiation therapy treatment, which may take several weeks, a patient's anatomy may change drastically, and adaptive radiation therapy (ART) may be needed. ...Cone-beam computed tomography (CBCT), which is often available during the treatment process, can be used for both patient positioning and ART re-planning. However, due to the prominent amount of noise, artifacts, and inaccurate Hounsfield unit (HU) values, the dose calculation based on CBCT images could be inaccurate for treatment planning. One way to solve this problem is to convert CBCT images to more accurate synthesized CT (sCT) images. In this work, we have developed a cycle-consistent generative adversarial network framework (CycleGAN) to synthesize CT images from CBCT images. This model is capable of image-to-image translation using unpaired CT and CBCT images in an unsupervised learning setting. The sCT images generated from CBCT through this CycleGAN model are visually and quantitatively similar to real CT images with decreased mean absolute error (MAE) from 69.29 HU to 29.85 HU for head-and-neck (H&N) cancer patients. The dose distributions calculated on the sCT by CycleGAN demonstrated a higher accuracy than those on CBCT in a 3D gamma index analysis with increased gamma index pass rate from 86.92% to 96.26% under 1 mm/1% criteria, when using the deformed planning CT image (dpCT) as the reference. We also compared the CycleGAN model with other unsupervised learning methods, including deep convolutional generative adversarial networks (DCGAN) and progressive growing of GANs (PGGAN), and demonstrated that CycleGAN outperformed the other two models. A phantom study has been conducted to compare sCT with dpCT, and the increase of structural similarity index from 0.91 to 0.93 shows that CycleGAN performed better than DIR in terms of preserving anatomical accuracy.
Display omitted
•Application of digital twin in life prediction of mechanical equipment.•Digital twin provide rich and independent virtual simulation data.•The improved CycleGAN can map simulation ...data in virtual space to measured data in physical space.•Large sample data can significantly improve the accuracy of various life prediction models.
The prediction accuracy of the remaining useful life of rolling bearings is greatly affected by the size of sample data, and it is difficult to obtain enough fault samples in practical applications. Digital twin technology can reproduce the operation process of rolling bearings and other mechanical equipment in the digital world, which provides a new paradigm for life prediction under the condition of small samples. In this paper, a virtual and real combination of life-cycle rolling bearing digital twin is proposed. The modified CycleGAN combined with Wasserstein distance is used to map the simulation data in virtual space to the measured data in physical space, which significantly reduces the error between the simulation data and the measured data. The effectiveness of the improved rolling bearing digital twin and the availability of simulation data are verified by experiments. The simulation data are applied to the advanced remaining useful life prediction method, and the high-precision life prediction of rolling bearings is realized. The comparison with the traditional life prediction method verifies that the proposed method can effectively solve the small sample problem.
The application of deep learning in the field of drug discovery brings the development and expansion of molecular generative models along with new challenges in this field. One of challenges in
de ...novo
molecular generation is how to produce new reasonable molecules with desired pharmacological, physical, and chemical properties. To improve the similarity between the generated molecule and the starting molecule, we propose a new molecule generation model by embedding Long Short-Term Memory (LSTM) and Attention mechanism in CycleGAN architecture, LA-CycleGAN. The network layer of the generator in CycleGAN is fused head and tail to improve the similarity of the generated structure. The embedded LSTM and Attention mechanism can overcome long-term dependency problems in treating the normally used SMILES input. From our quantitative evaluation, we present that LA-CycleGAN expands the chemical space of the molecules and improves the ability of structure conversion. The generated molecules are highly similar to the starting compound structures while obtaining expected molecular properties during cycle generative adversarial network learning, which comprehensively improves the performance of the generative model.
PDF
