论文标题
图像引导点云形状完成的跨模式学习
Cross-modal Learning for Image-Guided Point Cloud Shape Completion
论文作者
论文摘要
在本文中，我们在辅助图像的指导下探讨了点云完成的最新主题。我们展示了如何在局部潜在空间中有效地结合两种方式中的信息，从而避免了从最新的单个视图中使用复杂点云重建方法的需求。我们还研究了一种新颖的弱监督设置，其中辅助图像通过在完整的点云上使用可区分的渲染器来测量图像空间中的保真度，从而为训练过程提供了监督信号。实验显示了对单峰和多模式完成的最先进的监督方法的显着改善。我们还展示了弱监督的方法的有效性，该方法的表现优于许多监督方法，并且与最新的监督模型仅利用点云信息具有竞争力。
In this paper we explore the recent topic of point cloud completion, guided by an auxiliary image. We show how it is possible to effectively combine the information from the two modalities in a localized latent space, thus avoiding the need for complex point cloud reconstruction methods from single views used by the state-of-the-art. We also investigate a novel weakly-supervised setting where the auxiliary image provides a supervisory signal to the training process by using a differentiable renderer on the completed point cloud to measure fidelity in the image space. Experiments show significant improvements over state-of-the-art supervised methods for both unimodal and multimodal completion. We also show the effectiveness of the weakly-supervised approach which outperforms a number of supervised methods and is competitive with the latest supervised models only exploiting point cloud information.
