论文标题
评估野外可靠性评估的深泰勒分解
Evaluating Deep Taylor Decomposition for Reliability Assessment in the Wild
论文作者
论文摘要
我们认为，我们需要评估“野外”模型的可解释性方法，即在专业人士做出关键决策的情况下，模型可以潜在地帮助他们。我们提出了基于深层泰勒分解的代币归因的野外评估，专业记者进行可靠性评估。我们发现，在八卦语料库上进行了微调，将这种方法与罗伯塔·洛尔格（Roberta-Large）结合使用，导致了更快，更好的人类决策，以及对记者中新闻来源的更为批判性态度。我们介绍了人类和模型理由的比较，以及对记者在循环决策中的经历的定性分析。
We argue that we need to evaluate model interpretability methods 'in the wild', i.e., in situations where professionals make critical decisions, and models can potentially assist them. We present an in-the-wild evaluation of token attribution based on Deep Taylor Decomposition, with professional journalists performing reliability assessments. We find that using this method in conjunction with RoBERTa-Large, fine-tuned on the Gossip Corpus, led to faster and better human decision-making, as well as a more critical attitude toward news sources among the journalists. We present a comparison of human and model rationales, as well as a qualitative analysis of the journalists' experiences with machine-in-the-loop decision making.
