论文标题
考试：用于跨语性和多语言问题回答的多主体高中考试数据集
EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answering
论文作者
论文摘要
我们提出了考试 - 一个新的基准数据集，用于用于高中考试的跨语性和多语言问题。我们用16种语言收集了24,000多种高质量的高中考试问题，涵盖了8个语言家庭和24个来自自然科学和社会科学的学科。 考试提供了跨多种语言和主题的精细粒度评估框架，可以精确分析和比较各种模型。我们对现有表现最佳的多语言预训练模型进行了各种实验，我们表明考试提供了多种挑战，需要在多个领域中进行多语言知识和推理。我们希望考试能够使研究人员能够探索具有挑战性的推理和知识转移方法，并以各种不可能的语言回答学校问答的预培训模型。数据，代码，预训练模型和评估可在https://github.com/mhardalov/exams-qa上获得。
We propose EXAMS -- a new benchmark dataset for cross-lingual and multilingual question answering for high school examinations. We collected more than 24,000 high-quality high school exam questions in 16 languages, covering 8 language families and 24 school subjects from Natural Sciences and Social Sciences, among others. EXAMS offers a fine-grained evaluation framework across multiple languages and subjects, which allows precise analysis and comparison of various models. We perform various experiments with existing top-performing multilingual pre-trained models and we show that EXAMS offers multiple challenges that require multilingual knowledge and reasoning in multiple domains. We hope that EXAMS will enable researchers to explore challenging reasoning and knowledge transfer methods and pre-trained models for school question answering in various languages which was not possible before. The data, code, pre-trained models, and evaluation are available at https://github.com/mhardalov/exams-qa.
