HiSGAN: Image-to-Image Translation with Involution based Hybrid-Scale Transformer and Contrastive learning

Image-to-image translation is a highly challenging task, as it requires an accurate understanding of image details and their consistent transformation across domains. Notably, GANs have achieved remarkable success in this field. In essence, convolutional layers are the primary building blocks of these architectures. However, the limited receptive field in shallow layers makes it difficult to capture long-range spatial dependencies and non-local context. In this paper, the HiSGAN architecture is proposed to address this limitation. It combines deep representations with traditional techniques, such as SVD and Fast Fourier Convolution (FFC), to effectively extract style-related information and establish a global receptive field. Furthermore, we introduce the HiS-Transformer block with an involution operator in the bottleneck of the generator. This proposed block utilizes hybrid-scale self-attention to adaptively preserve the global receptive field and fine-grained information in salient regions while maintaining low computational cost. HiSGAN employs a new loss function based on gradient contrastive learning to improve cross-domain feature alignment. Quantitative and qualitative results on four public datasets demonstrate the superiority of the proposed approach over state-of-the-art methods. Importantly, these performance gains are achieved while reducing the parameter count and accelerating training. The code is available at https://github.com/OliverRensu/SG-Former

کلیدواژه ها:

Image-to-Image Translation ، Fast Fourier Convolution ، Transformers ، Contrastive Learning ، Feature extraction