论文标题
用自我监督的暹罗自动编码器在野外重新拍摄图像
Relighting Images in the Wild with a Self-Supervised Siamese Auto-Encoder
论文作者
论文摘要
我们提出了一种自我监督的方法，用于对野外的单个视图图像进行图像重新保留。该方法基于一个自动编码器，该自动编码器分别将图像分别分成两个单独的编码，分别与场景照明和内容有关。为了在不监督的情况下解散这些嵌入信息，我们利用了这样的假设，即某些增强操作不会影响图像内容，而仅影响光的方向。引入了一种新型的损耗函数，称为球形谐波损失，迫使嵌入光的照明将其转换为球形谐波矢量。我们在YouTube 8M和Celeba等大型数据集上训练模型。我们的实验表明，我们的方法可以正确估计场景照明，并在没有任何监督或先前的形状模型的情况下实际重新点亮输入图像。与监督方法相比，我们的方法具有相似的性能，并避免了常见的照明工件。
We propose a self-supervised method for image relighting of single view images in the wild. The method is based on an auto-encoder which deconstructs an image into two separate encodings, relating to the scene illumination and content, respectively. In order to disentangle this embedding information without supervision, we exploit the assumption that some augmentation operations do not affect the image content and only affect the direction of the light. A novel loss function, called spherical harmonic loss, is introduced that forces the illumination embedding to convert to a spherical harmonic vector. We train our model on large-scale datasets such as Youtube 8M and CelebA. Our experiments show that our method can correctly estimate scene illumination and realistically re-light input images, without any supervision or a prior shape model. Compared to supervised methods, our approach has similar performance and avoids common lighting artifacts.
