论文标题
制作智力：智商和ML基准中的道德价值观
Making Intelligence: Ethical Values in IQ and ML Benchmarks
论文作者
论文摘要
近年来，ML研究人员通过定义和改进机器学习（ML）基准和数据集进行了努力。同时，有些人对数据集创建和ML研究的伦理训练了关键的镜头。在该立场论文中，我们以``技术''或``科学''关于ML基准设计的决定的纠缠而来了。我们的起点是人类智能基准和ML基准之间存在多种被忽视的结构相似性。两种类型的基准都设定了用于描述，评估和比较与智能相关的任务的标准 - 许多人类智能学者长期以来都被认为是具有价值的标准。我们利用女性主义科学哲学对智商基准和社会科学中厚实的概念的观点来争辩说，创建ML基准时需要考虑和记录价值。通过创建价值中性基准来避免这种选择既不可能也不希望。最后，我们概述了ML基准研究伦理和道德评论的实用建议。
In recent years, ML researchers have wrestled with defining and improving machine learning (ML) benchmarks and datasets. In parallel, some have trained a critical lens on the ethics of dataset creation and ML research. In this position paper, we highlight the entanglement of ethics with seemingly ``technical'' or ``scientific'' decisions about the design of ML benchmarks. Our starting point is the existence of multiple overlooked structural similarities between human intelligence benchmarks and ML benchmarks. Both types of benchmarks set standards for describing, evaluating, and comparing performance on tasks relevant to intelligence -- standards that many scholars of human intelligence have long recognized as value-laden. We use perspectives from feminist philosophy of science on IQ benchmarks and thick concepts in social science to argue that values need to be considered and documented when creating ML benchmarks. It is neither possible nor desirable to avoid this choice by creating value-neutral benchmarks. Finally, we outline practical recommendations for ML benchmark research ethics and ethics review.
