论文标题
通过无监督的学习从甘恩中提取语义知识
Extracting Semantic Knowledge from GANs with Unsupervised Learning
论文作者
论文摘要
最近，无监督的学习在各种任务上取得了令人印象深刻的进步。尽管有歧视模型的主导地位，但越来越多的注意力吸引了由生成模型，尤其是生成对抗网络（GAN）学到的表示。关于gan的解释的先前作品表明，gans以线性可分离形式中的特征图中编码语义。在这项工作中，我们进一步发现，GAN的特征可以通过线性可分离性假设很好地聚集。我们提出了一种新颖的聚类算法，名为Klish，该算法利用了线性的可分离性来群集的特征。 Klish成功地提取了在各种物体（例如汽车，肖像，动物等）数据集中训练的gan的细颗粒语义。借助Klish，我们可以从GAN的图像及其分割掩码中采样图像并合成配对的图像分割数据集。使用合成的数据集，我们启用了两个下游应用程序。首先，我们在这些数据集上训练语义分割网络，并在真实图像上测试它们，从而实现无监督的语义细分。其次，我们在合成数据集上训练图像到图像翻译网络，从而实现没有人类注释的语义条件图像综合。
Recently, unsupervised learning has made impressive progress on various tasks. Despite the dominance of discriminative models, increasing attention is drawn to representations learned by generative models and in particular, Generative Adversarial Networks (GANs). Previous works on the interpretation of GANs reveal that GANs encode semantics in feature maps in a linearly separable form. In this work, we further find that GAN's features can be well clustered with the linear separability assumption. We propose a novel clustering algorithm, named KLiSH, which leverages the linear separability to cluster GAN's features. KLiSH succeeds in extracting fine-grained semantics of GANs trained on datasets of various objects, e.g., car, portrait, animals, and so on. With KLiSH, we can sample images from GANs along with their segmentation masks and synthesize paired image-segmentation datasets. Using the synthesized datasets, we enable two downstream applications. First, we train semantic segmentation networks on these datasets and test them on real images, realizing unsupervised semantic segmentation. Second, we train image-to-image translation networks on the synthesized datasets, enabling semantic-conditional image synthesis without human annotations.
