论文标题
具有神经递延阴影的多视图网状重建
Multi-View Mesh Reconstruction with Neural Deferred Shading
论文作者
论文摘要
我们提出了一种通过任意材料和照明的不透明对象的快速多视图3D重建的分析方法。最先进的方法同时使用神经表面表示和神经渲染。虽然灵活，但神经表面表示是优化运行时的重要瓶颈。取而代之的是，我们表示表面为三角形，并围绕三角栅格化和神经阴影构建可区分的渲染管道。渲染器用于梯度下降优化，其中三角形网格和神经着色器均已共同优化以重现多视图图像。我们在公共3D重建数据集上评估我们的方法，并表明它可以与传统基线和神经方法的重建精度相匹配，同时在优化运行时超过它们。此外，我们研究着着色器并发现它学习了外观的可解释表示，从而使应用程序（例如3D材料编辑）能够。
We propose an analysis-by-synthesis method for fast multi-view 3D reconstruction of opaque objects with arbitrary materials and illumination. State-of-the-art methods use both neural surface representations and neural rendering. While flexible, neural surface representations are a significant bottleneck in optimization runtime. Instead, we represent surfaces as triangle meshes and build a differentiable rendering pipeline around triangle rasterization and neural shading. The renderer is used in a gradient descent optimization where both a triangle mesh and a neural shader are jointly optimized to reproduce the multi-view images. We evaluate our method on a public 3D reconstruction dataset and show that it can match the reconstruction accuracy of traditional baselines and neural approaches while surpassing them in optimization runtime. Additionally, we investigate the shader and find that it learns an interpretable representation of appearance, enabling applications such as 3D material editing.
