论文标题
Pixelnerf：来自一个或几个图像的神经辐射场
pixelNeRF: Neural Radiance Fields from One or Few Images
论文作者
论文摘要
我们提出了Pixelnerf，这是一个学习框架，可以预测以一个或几个输入图像为条件的连续神经场景表示。现有的构建神经辐射场的方法涉及独立优化每个场景的表示形式，需要许多校准的视图和大量的计算时间。我们通过以完全卷积的方式引入nerf在图像输入上的结构来解决这些缺点，以解决这些缺点。这使网络可以在多个场景上进行训练，以便先验学习场景，从而使其能够从一组稀疏的视图（少于一个）中以馈送方式进行新颖的视图合成。利用NERF的音量渲染方法，我们的模型可以直接从图像中直接培训，而无需显式3D监督。我们对具有固定对象以及整个看不见类别的单一图像视图综合任务进行塑形基准测试的大量实验。我们通过在DTU数据集中的多对象Shapenet场景和真实场景上演示Pixelnerf的灵活性。在所有情况下，Pixelnerf的表现都均优于新型视图合成和单个图像3D重建的当前最新基准。有关视频和代码，请访问项目网站：https：//alexyu.net/pixelnerf
We propose pixelNeRF, a learning framework that predicts a continuous neural scene representation conditioned on one or few input images. The existing approach for constructing neural radiance fields involves optimizing the representation to every scene independently, requiring many calibrated views and significant compute time. We take a step towards resolving these shortcomings by introducing an architecture that conditions a NeRF on image inputs in a fully convolutional manner. This allows the network to be trained across multiple scenes to learn a scene prior, enabling it to perform novel view synthesis in a feed-forward manner from a sparse set of views (as few as one). Leveraging the volume rendering approach of NeRF, our model can be trained directly from images with no explicit 3D supervision. We conduct extensive experiments on ShapeNet benchmarks for single image novel view synthesis tasks with held-out objects as well as entire unseen categories. We further demonstrate the flexibility of pixelNeRF by demonstrating it on multi-object ShapeNet scenes and real scenes from the DTU dataset. In all cases, pixelNeRF outperforms current state-of-the-art baselines for novel view synthesis and single image 3D reconstruction. For the video and code, please visit the project website: https://alexyu.net/pixelnerf
