论文标题
Unicausal：因果文本挖掘的统一基准和存储库
UniCausal: Unified Benchmark and Repository for Causal Text Mining
论文作者
论文摘要
当前的因果文本挖掘数据集在目标，数据覆盖率和注释方案中有所不同。这些不一致的努力阻止了建模能力和模型性能的公平比较。此外，很少有数据集包含因果跨度注释，这是端到端因果关系提取所需的。为了解决这些问题，我们提出了UNICAUSAL，这是跨三个任务的因果文本开采的统一基准：（i）因果序列分类，（ii）因果效应跨度检测和（iii）因果对分类。我们合并了六个高质量的注释，主要是人类通知的语料库，分别为每个任务提供了58,720、12,144和69,165个示例。由于因果关系的定义可以是主观的，因此我们的框架旨在允许研究人员从事某些或所有数据集和任务。为了创建初始的基准测试，我们为每个任务微调了BERT预训练的语言模型，分别实现了70.10％的二进制F1、52.42％宏F1和84.68％的二进制F1分数。
Current causal text mining datasets vary in objectives, data coverage, and annotation schemes. These inconsistent efforts prevent modeling capabilities and fair comparisons of model performance. Furthermore, few datasets include cause-effect span annotations, which are needed for end-to-end causal relation extraction. To address these issues, we propose UniCausal, a unified benchmark for causal text mining across three tasks: (I) Causal Sequence Classification, (II) Cause-Effect Span Detection and (III) Causal Pair Classification. We consolidated and aligned annotations of six high quality, mainly human-annotated, corpora, resulting in a total of 58,720, 12,144 and 69,165 examples for each task respectively. Since the definition of causality can be subjective, our framework was designed to allow researchers to work on some or all datasets and tasks. To create an initial benchmark, we fine-tuned BERT pre-trained language models to each task, achieving 70.10% Binary F1, 52.42% Macro F1, and 84.68% Binary F1 scores respectively.
