论文标题
使用图形变压器的零拍素描图像检索
Zero-Shot Sketch Based Image Retrieval using Graph Transformer
论文作者
论文摘要
基于零拍的图像检索（ZS-SBIR）任务的性能主要受两个挑战的影响。图像和草图功能之间的实质域间隙需要桥接，同时必须巧妙地选择侧面信息。现有文献表明，改变语义方面的信息会极大地影响ZS-SBIR的性能。为此，我们提出了一种基于新的图形变压器基于零绘制的图像检索（GTZSR）框架，用于求解ZS-SBIR任务，该框架使用新颖的图形变压器在语义空间中保留类的拓扑，并在视觉空间的嵌入式特征中传播类的上下文图。为了弥合视觉特征之间的域间隙，我们建议最大程度地降低博学的域共享空间中图像和草图之间的瓦斯汀距离。我们还提出了一种新颖的兼容性损失，该损失通过桥接一个类别的域间隙相对于训练集中所有其他类别的域间隙，进一步使两个视觉域。在扩展的粗略，Tu-Berlin和QuickDraw数据集上获得的实验结果对ZS-SBIR和广义ZS-SBIR的现有最新方法表现出急剧的改进。
The performance of a zero-shot sketch-based image retrieval (ZS-SBIR) task is primarily affected by two challenges. The substantial domain gap between image and sketch features needs to be bridged, while at the same time the side information has to be chosen tactfully. Existing literature has shown that varying the semantic side information greatly affects the performance of ZS-SBIR. To this end, we propose a novel graph transformer based zero-shot sketch-based image retrieval (GTZSR) framework for solving ZS-SBIR tasks which uses a novel graph transformer to preserve the topology of the classes in the semantic space and propagates the context-graph of the classes within the embedding features of the visual space. To bridge the domain gap between the visual features, we propose minimizing the Wasserstein distance between images and sketches in a learned domain-shared space. We also propose a novel compatibility loss that further aligns the two visual domains by bridging the domain gap of one class with respect to the domain gap of all other classes in the training set. Experimental results obtained on the extended Sketchy, TU-Berlin, and QuickDraw datasets exhibit sharp improvements over the existing state-of-the-art methods in both ZS-SBIR and generalized ZS-SBIR.
