Astraea: Grammar-Based Fairness Testing
Astraea: Grammar-Based Fairness Testing
复制标题
Astraea：基于语法的公平性测试
DOI：
10.1109/tse.2022.3141758
复制
发表时间：
2020
影响因子：
7.4
通讯作者：
Sudipta Chattopadhyay
中科院分区：
文献类型：
--
作者：
E. Soremekun;Sakshi Udeshi;Sudipta Chattopadhyay
软件通常会产生偏见的输出。 ，亚马逊，微软和Google提供了产生不公平输出的软件服务，主要是由于社交偏见（例如性别或种族）。对进行<Italic>公平测试的任务</iTalic>。基于公平的测试方法</italic>（称为<sc> astraea </sc>），利用无上下文的语法生成<Italic>在软件系统中揭示公平侵犯的歧视性输入概率的语法，<sc> astraea </sc>还通过<Italic>隔离观察到的软件偏见的原因</italic> SC> Astraea </sc>在我们的评估中提供了三种主要的<Italic>自然语言处理的软件系统，该系统在我们的评估中以<sc> astraea </sc>的速度进行了公平侵犯。约18％。 <sc> Astraea </sc>在573K歧视性测试案例上产生了超过102k的公平性，<sc> astraea </sc>通过平均而言，通过型号将软件公平提高了约76％。
Software often produces biased outputs. In particular, machine learning (ML) based software is known to produce erroneous predictions when processing <italic>discriminatory inputs</italic>. Such unfair program behavior can be caused by societal bias. In the last few years, Amazon, Microsoft and Google have provided software services that produce unfair outputs, mostly due to societal bias (e.g., gender or race). In such events, developers are saddled with the task of conducting <italic>fairness testing</italic>. Fairness testing is challenging; developers are tasked with <italic>generating discriminatory inputs that reveal and explain biases</italic>. We propose a <italic>grammar-based fairness testing approach</italic> (called <sc>Astraea</sc>) which leverages context-free grammars to generate discriminatory inputs that <italic>reveal fairness violations</italic> in software systems. Using probabilistic grammars, <sc>Astraea</sc> also provides fault diagnosis by <italic>isolating the cause</italic> of observed software bias. <sc>Astraea</sc>’s diagnoses facilitate the improvement of ML fairness. <sc>Astraea</sc> was evaluated on 18 software systems that provide three major <italic>natural language processing</italic> (NLP) services. In our evaluation, <sc>Astraea</sc> generated fairness violations at a rate of about 18%. <sc>Astraea</sc> generated over 573K discriminatory test cases and found over 102K fairness violations. Furthermore, <sc>Astraea</sc> improves software fairness by about 76% via model-retraining, on average.
登录
查看更多内容
NLP 中的种族、种族主义和反种族主义调查
DOI：
10.18653/v1/2021.acl-long.149
发表时间：
2021
期刊：
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers
影响因子：
--
作者：
Field, Anjalie;Blodgett, Su Lin;Waseem, Zeerak;Tsvetkov, Yulia
通讯作者：
Tsvetkov, Yulia
DOI：
10.18653/v1/p19-1159
发表时间：
2019-06
期刊：
--
影响因子：
--
作者：
Tony Sun;Andrew Gaut;Shirlyn Tang;Yuxin Huang;Mai Elsherief;Jieyu Zhao;Diba Mirza;E. Belding-Royer;Kai-Wei Chang;William Yang Wang
通讯作者：
Tony Sun;Andrew Gaut;Shirlyn Tang;Yuxin Huang;Mai Elsherief;Jieyu Zhao;Diba Mirza;E. Belding-Royer;Kai-Wei Chang;William Yang Wang
DOI：
10.18653/v1/n18-2003
发表时间：
2018-04
期刊：
ArXiv
影响因子：
--
作者：
Jieyu Zhao;Tianlu Wang;Mark Yatskar;Vicente Ordonez;Kai-Wei Chang
通讯作者：
Jieyu Zhao;Tianlu Wang;Mark Yatskar;Vicente Ordonez;Kai-Wei Chang
影响因子：
1.8
作者：
Albarghouthi, Aws;D'Antoni, Loris;Nori, Aditya, V
通讯作者：
Nori, Aditya, V
DOI：
10.1109/icse43902.2021.00032
发表时间：
2021-02
期刊：
2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)
影响因子：
--
作者：
Swaroopa Dola;Matthew B. Dwyer;M. Soffa
通讯作者：
Swaroopa Dola;Matthew B. Dwyer;M. Soffa