论文标题
及时调整生成多模式预审预周座模型
Prompt Tuning for Generative Multimodal Pretrained Models
论文作者
论文摘要
及时的调整已成为模型调整的新范式，并且在自然语言进行了预训练甚至训练时表现出成功。在这项工作中，我们探讨了迅速调整到多模式预处理的转移，重点是生成的多模式预验证的模型，而不是对比度的模型。具体而言，我们实施了迅速调整统一的序列到序列预测的模型，以适应理解和生成任务。实验结果表明，轻重量提示调整可以通过填充并超过其他轻量调整方法来实现可比的性能。此外，与固定模型相比，迅速调整的模型表明了针对对抗性攻击的鲁棒性。我们进一步确定，实验因素，包括及时长度，及时的深度和重新聚集化，对模型性能产生了很大的影响，因此我们从经验上为迅速调整的设置提供了建议。尽管有观察到的优势，但我们仍然在迅速调整中发现了一些局限性，我们相应地指出了未来研究的方向。代码可在\ url {https://github.com/ofa-sys/ofa}中获得
Prompt tuning has become a new paradigm for model tuning and it has demonstrated success in natural language pretraining and even vision pretraining. In this work, we explore the transfer of prompt tuning to multimodal pretraining, with a focus on generative multimodal pretrained models, instead of contrastive ones. Specifically, we implement prompt tuning on the unified sequence-to-sequence pretrained model adaptive to both understanding and generation tasks. Experimental results demonstrate that the light-weight prompt tuning can achieve comparable performance with finetuning and surpass other light-weight tuning methods. Besides, in comparison with finetuned models, the prompt-tuned models demonstrate improved robustness against adversarial attacks. We further figure out that experimental factors, including the prompt length, prompt depth, and reparameteratization, have great impacts on the model performance, and thus we empirically provide a recommendation for the setups of prompt tuning. Despite the observed advantages, we still find some limitations in prompt tuning, and we correspondingly point out the directions for future studies. Codes are available at \url{https://github.com/OFA-Sys/OFA}
