论文标题
变分的开放域问题回答
Variational Open-Domain Question Answering
论文作者
论文摘要
被证明在自然语言处理任务中有效检索模型，但使用变分推断对其优化仍缺乏研究。我们介绍了端到端培训的变异开放域（VOD）框架，并评估检索型模型，重点关注开放域的问题答案和语言建模。 VOD物镜是对rényi变分结合的自称的估计值，近似于任务边际可能性，并根据从辅助采样分布（缓存的检索器和/或近似后验）进行的样品进行评估。即使对于大型语料库中定义的回猎犬分布，它仍然可以进行处理。我们通过在多项选择体检问题上培训阅读器 - 重新者大小的模型来证明VOD的多功能性。在MEDMCQA数据集上，尽管使用了2.500 $ \ tims $ $ $ $ $，但我们的表现均优于域调节的Med-Palm +5.3％。我们的检索型Biolinkbert模型在MEDMCQA上得分为62.9％，MEDQA-USMLE得分为55.0％。最后，我们在医学语义搜索的背景下展示了我们学到的猎犬组成部分的有效性。
Retrieval-augmented models have proven to be effective in natural language processing tasks, yet there remains a lack of research on their optimization using variational inference. We introduce the Variational Open-Domain (VOD) framework for end-to-end training and evaluation of retrieval-augmented models, focusing on open-domain question answering and language modelling. The VOD objective, a self-normalized estimate of the Rényi variational bound, approximates the task marginal likelihood and is evaluated under samples drawn from an auxiliary sampling distribution (cached retriever and/or approximate posterior). It remains tractable, even for retriever distributions defined on large corpora. We demonstrate VOD's versatility by training reader-retriever BERT-sized models on multiple-choice medical exam questions. On the MedMCQA dataset, we outperform the domain-tuned Med-PaLM by +5.3% despite using 2.500$\times$ fewer parameters. Our retrieval-augmented BioLinkBERT model scored 62.9% on the MedMCQA and 55.0% on the MedQA-USMLE. Last, we show the effectiveness of our learned retriever component in the context of medical semantic search.
