论文标题
推断：西班牙语的自然语言推理语料库，具有基于否定的对比和对抗性示例
InferES : A Natural Language Inference Corpus for Spanish Featuring Negation-Based Contrastive and Adversarial Examples
论文作者
论文摘要
在本文中，我们提出了推论 - 欧洲西班牙语的原始自然语言推断（NLI）。我们提出，实施和分析利用专家语言学家和人群工人的各种语料库创造策略。推论背后的目标是提供高质量的数据，同时促进自动化系统的系统评估。具体而言，我们专注于衡量和改善机器学习系统在基于否定的对抗示例上的性能及其在跨分布主题中概括的能力。 在各种情况下，我们在推断（8,055个黄金示例）上训练两个变压器模型。我们的最佳模型获得了72.8％的精度，留出了很多改进的空间。 “仅假设”基线仅比大多数高2％-5％的效果，表明注释伪像比先前的工作要少得多。我们发现，经过培训的推断训练的模型在跨主题（包括分布和分布的情况下）都非常好，并且在基于否定的对抗示例上表现中等。
In this paper, we present InferES - an original corpus for Natural Language Inference (NLI) in European Spanish. We propose, implement, and analyze a variety of corpus-creating strategies utilizing expert linguists and crowd workers. The objectives behind InferES are to provide high-quality data, and, at the same time to facilitate the systematic evaluation of automated systems. Specifically, we focus on measuring and improving the performance of machine learning systems on negation-based adversarial examples and their ability to generalize across out-of-distribution topics. We train two transformer models on InferES (8,055 gold examples) in a variety of scenarios. Our best model obtains 72.8% accuracy, leaving a lot of room for improvement. The "hypothesis-only" baseline performs only 2%-5% higher than majority, indicating much fewer annotation artifacts than prior work. We find that models trained on InferES generalize very well across topics (both in- and out-of-distribution) and perform moderately well on negation-based adversarial examples.
