论文标题
视觉提示调整测试时间域的适应
Visual Prompt Tuning for Test-time Domain Adaptation
论文作者
论文摘要
模型应该能够在测试时间内适应看不见的数据，以避免由现实部署场景中不可避免的分配变化引起的性能下降。在这项工作中，我们解决了实用但具有挑战性的测试时间适应（TTA）问题，其中模型在不访问源数据的情况下适应了目标域。我们提出了一个简单的食谱，称为\ textit {数据有效提示调整}（dept），带有两个关键成分。首先，部门将视觉提示插入视觉变压器，仅在适应过程中调整这些源定义的提示。我们发现，这种参数有效的列式可以有效地将模型表示形式调整到目标域而不适合学习目标中的噪声。其次，部门通过基于内存的在线伪标记来引导源代表到目标域。专门为提示设计的分层自我监督正规化联合优化以减轻自训练期间的错误积累。在可调的参数较少的情况下，部门不仅展示了主要适应基准Visda-C，ImageNet-C和DomainNet-126上的最先进性能，而且还表现出卓越的数据效率，即仅使用1 \％或10 \％的数据适应，而没有太多的性能降级，而不是100 \％的数据。此外，部门还将扩展到在线或多源TTA设置。
Models should be able to adapt to unseen data during test-time to avoid performance drops caused by inevitable distribution shifts in real-world deployment scenarios. In this work, we tackle the practical yet challenging test-time adaptation (TTA) problem, where a model adapts to the target domain without accessing the source data. We propose a simple recipe called \textit{Data-efficient Prompt Tuning} (DePT) with two key ingredients. First, DePT plugs visual prompts into the vision Transformer and only tunes these source-initialized prompts during adaptation. We find such parameter-efficient finetuning can efficiently adapt the model representation to the target domain without overfitting to the noise in the learning objective. Second, DePT bootstraps the source representation to the target domain by memory bank-based online pseudo-labeling. A hierarchical self-supervised regularization specially designed for prompts is jointly optimized to alleviate error accumulation during self-training. With much fewer tunable parameters, DePT demonstrates not only state-of-the-art performance on major adaptation benchmarks VisDA-C, ImageNet-C, and DomainNet-126, but also superior data efficiency, i.e., adaptation with only 1\% or 10\% data without much performance degradation compared to 100\% data. In addition, DePT is also versatile to be extended to online or multi-source TTA settings.
