GHOST 2.0: Generative High-fidelity One Shot Transfer of Heads

Alexander Groshev, Anastasiia Iashchenko, Pavel Paramonov, Denis Dimitrov, Andrey Kuznetsov 
Sber AI, AIRI  
arXiv Code Generate

Abstract

While the task of face swapping has recently gained attention in the research community, a related problem of head swapping remains largely unexplored. In addition to skin color transfer, head swap poses extra challenges, such as the need to preserve structural information of the whole head during synthesis and inpaint gaps between swapped head and background. In this paper, we address these concerns with GHOST 2.0, which consists of two problem-specific modules. First, we introduce enhanced Aligner model for head reenactment, which preserves identity information at multiple scales and is robust to extreme pose variations. Secondly, we use a Blender module that seamlessly integrates the reenacted head into the target background by transferring skin color and inpainting mismatched regions. Both modules outperform the baselines on the corresponding tasks, allowing to achieve state-of-the-art results in head swapping. We also tackle complex cases, such as large difference in hair styles of source and target.

Overall Pipeline

Interpolate start reference image.

Our model pipeline include following parts:

  1. Aligner. To perform cross-reenactment by transfering target motion to source head, we have implemented new state-of-the-art level head reenactment model.
  2. Blender. To blend reenacted head into target portrait, we have implemented Blender module similar to one implemented in HeSer-like with usage of LaMa.
  3. Post-processing blending. To remove excess hair from image in the case of its extension beyond the boundaries of target crop, we optionally apply Kandinsky inpainting model to inpainting region.

BibTeX

@misc{groshev2025ghost2
      title={GHOST 2.0: generative high-fidelity one shot transfer of heads}, 
      author={Alexander Groshev and Anastasiia Iashchenko and Pavel Paramonov and Denis Dimitrov and Andrey Kuznetsov},
      year={2025},
      eprint={2502.18417},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2502.18417}, 
}