UP - logo
E-resources
Full text
Peer reviewed
  • Foreground and background s...
    Yu, Yue; Wang, Jianming; Li, Nengli

    Image and vision computing, March 2024, 2024-03-00, Volume: 143
    Journal Article

    Traditional image-based style transfer requires additional reference style images, making it less user-friendly. Text-based methods are more convenient but suffer from issues like slow generation, unclear content, and poor quality. In this work, we propose a new style transfer method SA2-CS (means Semantic-Aware and Salient Attention CLIPStyler), which is based on the Comparative Language Image Pretraining (CLIP) model and a salient object detection network. Masks obtained from the salient object detection network are utilized to guide the style transfer process, and various strategies are employed to optimize according to different masks. Adequate experiments with diverse content images and style text descriptions were conducted, demonstrating our method's advantages: the network is easily trainable and converges rapidly; it achieves stable, superior generation results compared to other methods. Our approach addresses over-stylization issues in the foreground, enhances foreground-background contrast, and enables precise control over style transfer in various semantic regions. •A Semantic-Aware and Salient Attention CLIPStyler is proposed for solving the task of text-based style transfer.•The method introduces U2-Net as the salient object detection network to realize the different degrees' stylization.•Semantic-aware PatchCLIP loss is proposed to solve the problem of poor output image effect.•Global background loss function and a mask are proposed to ensure that the background and foreground are not distorted.