论文标题
评估大语言模型的文本到SQL功能
Evaluating the Text-to-SQL Capabilities of Large Language Models
论文作者
论文摘要
我们对法典语言模型的文本到SQL功能进行经验评估。我们发现，在没有任何填充的情况下，Codex是蜘蛛基准的强大基线。我们还分析了在此设置中法典的故障模式。此外，我们在地球和学者的基准上证明了提示中提供的少数域内示例使Codex比在如此少的示例上进行了验证的最新模型更好。
We perform an empirical evaluation of Text-to-SQL capabilities of the Codex language model. We find that, without any finetuning, Codex is a strong baseline on the Spider benchmark; we also analyze the failure modes of Codex in this setting. Furthermore, we demonstrate on the GeoQuery and Scholar benchmarks that a small number of in-domain examples provided in the prompt enables Codex to perform better than state-of-the-art models finetuned on such few-shot examples.
