Generative Domain Adaptation (GDA) is a machine learning technique used to adapt a model trained in one domain (source) using a few examples from another domain (target). This is beneficial in situations where it is expensive or impractical to obtain substantial labeled data from the target domain. While existing GDA solutions focus on enhancing a specific AI model called a generator, to produce high-quality and diverse data resembling the target domain, obstacles arise when the source and target domains significantly differ, making it difficult for the generator to adapt.
A recent paper presented at NeurIPS called Domain Re-Modulation (DoRM) aims to overcome these challenges. Unlike previous methods, DoRM improves the quality and diversity of image synthesis and the consistency of cross-domain while integrating memory and domain association capabilities. DoRM does this by altering the style space using new mapping and affine modules allowing the generation of high-quality images across numerous domains, including those not seen during training.
DoRM enhances the generator’s capabilities through several key methods:
1. It begins with a pre-trained StyleGAN2 generator to serve as a foundation for future adaptations.
2. The source generator is frozen to adapt to the target domain, and new mapping and affine (M&A) modules are introduced to capture specific attributes unique to the target domain.
3. It adjusts the style space of the target domain’s latent code, enabling the generator to synthesize outputs that reflect the target domain’s characteristics accurately.
4. DoRM makes a linearly adjustable domain shift in the generator’s style space using M&A modules. It allows for precise adjustments for specific domains, enhancing the generator’s versatility and seamless blending of attributes from multiple training sources.
5. DoRM introduces a novel similarity-based structure loss to ensure consistency across domains using CLIP image encoder tokens.
6. It integrates an inclusive loss function that combines StyleGAN2’s original adversarial loss during training, optimizing generator and discriminator learning.
The research team applied the proposed method to the Flickr-Faces-HQ Dataset (FFHQ) using a pre-trained StyleGAN2 model and proved it to perform superiorly compared with other methods.
In conclusion, the researchers introduce DoRM as a streamlined generator structure designed for GDA. DoRM incorporates a novel similarity-based structure loss to ensure robust cross-domain consistency and offers superior synthesis quality, diversity, and cross-domain consistency compared to existing approaches. DoRM can generate images in hybrid domains not encountered during training, much like the human brain. The method has implications for future work in machine learning and domain adaptation.
