论文标题
我可以相信我的公平指标吗？通过未标记的数据和贝叶斯推断评估公平性
Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference
论文作者
论文摘要
当标记的例子很少，但未标记的例子很多时，我们研究了可靠评估群体公平性的问题。我们提出了一个通用的贝叶斯框架，该框架可以通过基于标记的数据来增强使用未标记数据的数据来增加标记的数据，以产生更准确和较低的变化估计。我们的方法估计了每组未标记的示例校准得分，使用标记示例的层次潜在变量模型。反过来，这允许对各种群体公平指标的不确定性概念进行后验分布。我们证明我们的方法会导致多个众所周知的公平数据集，敏感属性和预测模型的估计误差显着降低。结果表明，在评估预测模型是否合理的角度方面，同时使用未标记的数据和贝叶斯推断的好处。
We investigate the problem of reliably assessing group fairness when labeled examples are few but unlabeled examples are plentiful. We propose a general Bayesian framework that can augment labeled data with unlabeled data to produce more accurate and lower-variance estimates compared to methods based on labeled data alone. Our approach estimates calibrated scores for unlabeled examples in each group using a hierarchical latent variable model conditioned on labeled examples. This in turn allows for inference of posterior distributions with associated notions of uncertainty for a variety of group fairness metrics. We demonstrate that our approach leads to significant and consistent reductions in estimation error across multiple well-known fairness datasets, sensitive attributes, and predictive models. The results show the benefits of using both unlabeled data and Bayesian inference in terms of assessing whether a prediction model is fair or not.
