论文标题
指标的问题是AI的基本问题
The Problem with Metrics is a Fundamental Problem for AI
论文作者
论文摘要
优化给定的指标是大多数当前AI方法的中心方面，但是过分强调指标会导致操纵，游戏，近视关注短期目标以及其他意外的负面后果。这对人工智能发展构成了根本的矛盾。通过一系列现实世界中的案例研究，我们研究了指标在实践中出错的各个方面以及我们在线环境和当前业务实践如何加剧这些失败的方面。最后，我们提出了一个框架，以减轻AI中指标过度强调造成的危害：（1）使用一组指标来获得更全面，更细微的图片，（2）将指标与定性账户相结合，以及（3）涉及一系列利益相关者，包括那些将受到最大影响的利益相关者。
Optimizing a given metric is a central aspect of most current AI approaches, yet overemphasizing metrics leads to manipulation, gaming, a myopic focus on short-term goals, and other unexpected negative consequences. This poses a fundamental contradiction for AI development. Through a series of real-world case studies, we look at various aspects of where metrics go wrong in practice and aspects of how our online environment and current business practices are exacerbating these failures. Finally, we propose a framework towards mitigating the harms caused by overemphasis of metrics within AI by: (1) using a slate of metrics to get a fuller and more nuanced picture, (2) combining metrics with qualitative accounts, and (3) involving a range of stakeholders, including those who will be most impacted.
