What's Fair is Fair: Detecting and Mitigating Encoded Bias in Multimodal Models of Museum Visitor Attention
What's Fair is Fair: Detecting and Mitigating Encoded Bias in Multimodal Models of Museum Visitor Attention
复制标题
DOI：
10.1145/3462244.3479943
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Halim Acosta;Nathan L. Henderson;Jonathan P. Rowe;Wookhee Min;James Minogue;James C. Lester
中科院分区：
文献类型：
--
作者：
Halim Acosta;Nathan L. Henderson;Jonathan P. Rowe;Wookhee Min;James Minogue;James C. Lester
近年来，人们对用多模式学习分析来模拟博物馆的游客参与越来越感兴趣。与此同时，人们也越来越关注机器学习模型中的公平性和编码偏见问题。在本文中，我们研究了偏见检测和缓解技术，以解决博物馆游客视觉注意的多模态模型中的算法公平性问题。我们使用绝对roc间面积（ABROCA）统计数据进行切片分析，以检测多模态模型中存在的编码偏差，这些模型使用来自游客与基于游戏的博物馆展览有关环境可持续性的互动的面部表情和姿势数据进行训练。我们研究了在几种机器学习技术的不同模式组合之间产生的性别偏见的实例。我们还测量了两种不同的去偏策略——学习公平表征和重加权——在应用于训练好的多模式访问者注意力模型时的有效性。结果表明，对于不同的访问者视觉注意模型，不同的模态组合会产生不同的偏差模式，并且在预测准确性和ABROCA之间往往存在固有的权衡。分析表明，在访问者视觉注意的多模态模型中，去偏策略往往比单模态模型更有效
Recent years have seen growing interest in modeling visitor engagement in museums with multimodal learning analytics. In parallel, there has also been growing concern about issues of fairness and encoded bias in machine learning models. In this paper, we investigate bias detection and mitigation techniques to address issues of algorithmic fairness in multimodal models of museum visitor visual attention. We employ slicing analysis using the Absolute Between-ROC Area (ABROCA) statistic to detect encoded bias present in multimodal models of visitor visual attention trained with facial expression and posture data from visitor interactions with a game-based museum exhibit about environmental sustainability. We investigate instances of gender bias that arise between different combinations of modalities across several machine learning techniques. We also measure the effectiveness of two different debiasing strategies—learned fair representations and reweighing—when applied to the trained multimodal visitor attention models. Results indicate that patterns of bias can arise across different modality combinations for the different visitor visual attention models, and there is often an inherent tradeoff between predictive accuracy and ABROCA. Analyses suggest that debiasing strategies tend to be more effective on multimodal models of visitor visual attention than their unimodal counterparts