论文标题
在甘恩的潜在空间中寻找神经面部重演的指示
Finding Directions in GAN's Latent Space for Neural Face Reenactment
论文作者
论文摘要
本文是在面部/头部重演中，目标是将目标面的面部姿势（3D头方向和表达）转移到源面部。以前的方法着重于学习嵌入网络以进行身份和伪造，这被证明是一项相当艰巨的任务，从而降低了生成的图像的质量。我们采用不同的方法，绕过了（微调）预训练的gan的训练，这些甘室能够产生高质量的面部图像。因为甘恩的特征是弱可控性，所以我们方法的核心是一种发现潜在gan空间中哪些方向负责控制面部姿势和表达变化的方法。我们提出了一条简单的管道，借助于3D形状模型来学习此类方向，该模型通过构造已经捕获了针对面部姿势，身份和表达的解开方向。此外，我们表明，通过将真实图像嵌入到gan潜在空间中，我们的方法可以成功地用于重新制定现实面孔。我们的方法具有几种有利的属性，包括使用单个源图像（一击）和跨人重演。我们的定性和定量结果表明，我们的方法通常会产生重演面孔的质量明显高于voxceleb1＆2的标准基准测试的最先进方法所产生的面孔。
This paper is on face/head reenactment where the goal is to transfer the facial pose (3D head orientation and expression) of a target face to a source face. Previous methods focus on learning embedding networks for identity and pose disentanglement which proves to be a rather hard task, degrading the quality of the generated images. We take a different approach, bypassing the training of such networks, by using (fine-tuned) pre-trained GANs which have been shown capable of producing high-quality facial images. Because GANs are characterized by weak controllability, the core of our approach is a method to discover which directions in latent GAN space are responsible for controlling facial pose and expression variations. We present a simple pipeline to learn such directions with the aid of a 3D shape model which, by construction, already captures disentangled directions for facial pose, identity and expression. Moreover, we show that by embedding real images in the GAN latent space, our method can be successfully used for the reenactment of real-world faces. Our method features several favorable properties including using a single source image (one-shot) and enabling cross-person reenactment. Our qualitative and quantitative results show that our approach often produces reenacted faces of significantly higher quality than those produced by state-of-the-art methods for the standard benchmarks of VoxCeleb1 & 2. Source code is available at: https://github.com/StelaBou/stylegan_directions_face_reenactment
