论文标题

训练和调整生成的神经辐射场,用于属性3D感知面部生成

Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generation

论文作者

Zhang, Jichao, Siarohin, Aliaksandr, Liu, Yahui, Tang, Hao, Sebe, Nicu, Wang, Wei

论文摘要

基于生成的神经辐射场(GNERF)基于3D感知的gan在制作高保真图像方面表现出了非凡的能力,同时保持了强大的3D一致性,尤其是面对面的一致性。但是,特定的现有模型优先考虑视图一致性而不是分离,从而在生成过程中导致语义或属性控制受到限制。尽管许多方法都探索了结合语义面具或利用3D形态模型(3DMM)先验的语义模型,但这些方法通常需要从头开始训练,并需要大量的计算开销。在本文中,我们提出了一种新颖的方法:一种有条件的GNERF模型,该模型将特定属性标签集成为输入,从而扩大了3D吸引生成模型的可控性和分离功能。我们的方法建立在预先训练的3D感知面部模型的基础上,我们引入了培训作为初始化和优化调整(TRIOT)方法,以训练有条件的归一化流量模块以实现面部属性编辑,然后优化潜在矢量以进一步提高属性编辑精度。我们的广泛实验证实了我们的模型的功效,展示了其具有增强视图一致性的高质量编辑的能力,同时维护非目标区域。我们的模型代码可在https://github.com/zhangqianhui/tt-gnerf上公开获得。

Generative Neural Radiance Fields (GNeRF)-based 3D-aware GANs have showcased remarkable prowess in crafting high-fidelity images while upholding robust 3D consistency, particularly face generation. However, specific existing models prioritize view consistency over disentanglement, leading to constrained semantic or attribute control during the generation process. While many methods have explored incorporating semantic masks or leveraging 3D Morphable Models (3DMM) priors to imbue models with semantic control, these methods often demand training from scratch, entailing significant computational overhead. In this paper, we propose a novel approach: a conditional GNeRF model that integrates specific attribute labels as input, thus amplifying the controllability and disentanglement capabilities of 3D-aware generative models. Our approach builds upon a pre-trained 3D-aware face model, and we introduce a Training as Init and Optimizing for Tuning (TRIOT) method to train a conditional normalized flow module to enable the facial attribute editing, then optimize the latent vector to improve attribute-editing precision further. Our extensive experiments substantiate the efficacy of our model, showcasing its ability to generate high-quality edits with enhanced view consistency while safeguarding non-target regions. The code for our model is publicly available at https://github.com/zhangqianhui/TT-GNeRF.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源