论文标题
Qampari：一个开放域问题，回答基准的问题，并带有多个段落的许多答案
QAMPARI: An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphs
论文作者
论文摘要
开放域问题回答（ODQA）的现有基准通常集中于可以从单个段落中提取答案的问题。相比之下，许多自然问题，例如“布鲁克林篮网起草什么球员？”有答案列表。回答此类问题需要在大型语料库中从许多段落中检索和阅读。我们介绍了ODQA基准的Qampari，问题答案是实体列表，分布在许多段落中。我们通过（a）通过Wikipedia的知识图和表中的多个答案创建了Qampari，（b）自动将答案与Wikipedia段落中的支持证据配对，以及（c）手动解释问题并验证每个答案。我们从检索和阅读家庭中训练ODQA模型，发现Qampari在通道检索和答案生成方面都具有挑战性，最多达到32.8的F1分数。我们的结果强调了开发ODQA模型的需求，该模型处理广泛的问题类型，包括单一和多回答问题。
Existing benchmarks for open-domain question answering (ODQA) typically focus on questions whose answers can be extracted from a single paragraph. By contrast, many natural questions, such as "What players were drafted by the Brooklyn Nets?" have a list of answers. Answering such questions requires retrieving and reading from many passages, in a large corpus. We introduce QAMPARI, an ODQA benchmark, where question answers are lists of entities, spread across many paragraphs. We created QAMPARI by (a) generating questions with multiple answers from Wikipedia's knowledge graph and tables, (b) automatically pairing answers with supporting evidence in Wikipedia paragraphs, and (c) manually paraphrasing questions and validating each answer. We train ODQA models from the retrieve-and-read family and find that QAMPARI is challenging in terms of both passage retrieval and answer generation, reaching an F1 score of 32.8 at best. Our results highlight the need for developing ODQA models that handle a broad range of question types, including single and multi-answer questions.
