In the field of Artificial Intelligence (AI) and Medicine, chest X-ray images are crucial for diagnosing various diseases. However, training AI models presents challenges, particularly with limited data or cases involving significant pathology or minor anomalies. To address the constraint of limited data, data augmentation has emerged as a popular technique in medical imaging. One promising approach to augmenting chest X-ray images involves leveraging text-to-image generation, which transforms textual disease descriptions into synthetic images. This technique effectively rectifies class imbalances and enhances the accuracy and reliability of AI models used in medical imaging applications. This study introduces a text-to-image generation architecture based on DF-GAN to augment chest X-ray images. The study aims to assess the impact of augmented data on the performance of two AI models, namely VGG16 and ResNet50, in a classification task. The experimentation is conducted on two challenging datasets, namely Chest X-rays from Indiana University and NIH Chest X-rays. The findings reveal that integrating text-to-image generated data enhances sensitivity by 2.1%, specificity by 1.9%, and AUC by 1.4%, while also mitigating overfitting during training across both datasets. These results underscore the potential of text-to-image generation in bolstering the accuracy and robustness of AI models employed in medical imaging tasks. •Proposed text-to-image generation using DF-GAN to augment chest X-ray images.•Demonstrated impact of augmented data on VGG16 and ResNet50 in classifying X-rays.•Evaluated technique on challenging datasets: Chest X-rays (IU) and NIH X-rays.•Augmented data improved sensitivity by 2.1% and specificity by 1.9% across datasets.•Text-to-image generation mitigated overfitting and enhanced accuracy in medical imaging.