论文标题
实时神经特征通过姿势引导的多层图像渲染
Real-Time Neural Character Rendering with Pose-Guided Multiplane Images
论文作者
论文摘要
我们提出了姿势引导的多层图像（MPI）合成，可以在具有逼真的质量的真实场景中呈现动画角色。我们使用便携式摄像头钻机来捕获多视图图像以及移动主题的驱动信号。我们的方法概括了图像到图像翻译范式，该范式将人姿势转换为3D场景表示形式 - 使用多视图将其作为监督捕获的MPI可以在自由视点中渲染。为了充分培养MPI的潜力，我们提出了深度自适应MPI，可以使用可变的曝光图像来学习，同时又有鲁棒的摄像机注册。我们的方法证明了与具有挑战性动作的角色的最先进方法相比，具有优势的小说视图合成质量。此外，提出的方法可以推广到训练姿势的新型组合，并且可以明确控制。我们的方法实现了这种表现力和动画的角色，实时呈现所有内容，是实用应用的有前途的解决方案。
We propose pose-guided multiplane image (MPI) synthesis which can render an animatable character in real scenes with photorealistic quality. We use a portable camera rig to capture the multi-view images along with the driving signal for the moving subject. Our method generalizes the image-to-image translation paradigm, which translates the human pose to a 3D scene representation -- MPIs that can be rendered in free viewpoints, using the multi-views captures as supervision. To fully cultivate the potential of MPI, we propose depth-adaptive MPI which can be learned using variable exposure images while being robust to inaccurate camera registration. Our method demonstrates advantageous novel-view synthesis quality over the state-of-the-art approaches for characters with challenging motions. Moreover, the proposed method is generalizable to novel combinations of training poses and can be explicitly controlled. Our method achieves such expressive and animatable character rendering all in real time, serving as a promising solution for practical applications.
