Quadratic metric elicitation for fairness and beyond
Quadratic metric elicitation for fairness and beyond
复制标题
DOI：
--
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
G. Hiranandani;Jatin Mathur;H. Narasimhan;Oluwasanmi Koyejo
中科院分区：
文献类型：
--
作者：
G. Hiranandani;Jatin Mathur;H. Narasimhan;Oluwasanmi Koyejo
度量引出是最近的一个框架，用于引出分类性能指标，该指标最能反映基于任务和上下文的隐含用户偏好。然而，可用的启发策略仅限于预测率的线性（或准线性）函数，这实际上限制了包括公平性在内的许多应用。本文开发了一种策略，用于引出由二次函数定义的更灵活的多类指标，旨在更好地反映人类的偏好。我们展示了它在推导基于二次违例的群体公平指标中的应用。我们的策略只需要相对偏好反馈，对噪声具有鲁棒性，并且实现了接近最优的查询复杂度。我们进一步扩展这个策略来引出多项式度量——从而扩大度量度量引出的用例。
Metric elicitation is a recent framework for eliciting classification performance metrics that best reflect implicit user preferences based on the task and context. However, available elicitation strategies have been limited to linear (or quasi-linear) functions of predictive rates, which can be practically restrictive for many applications including fairness. This paper develops a strategy for eliciting more flexible multiclass metrics defined by quadratic functions of rates, designed to reflect human preferences better. We show its application in eliciting quadratic violation-based group-fair metrics. Our strategy requires only relative preference feedback, is robust to noise, and achieves near-optimal query complexity. We further extend this strategy to eliciting polynomial metrics -- thus broadening the use cases for metric elicitation.