The Model Card Authoring Toolkit: Toward Community-centered, Deliberation-driven AI Design
The Model Card Authoring Toolkit: Toward Community-centered, Deliberation-driven AI Design
复制标题
模型卡片创作工具包：迈向以社区为中心、审议驱动的人工智能设计
DOI：
10.1145/3531146.3533110
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Zhu, Haiyi
中科院分区：
文献类型：
--
作者：
Shen, Hong;Wang, Leijie;Deng, Wesley H.;Brusse, Ciell;Velgersdijk, Ronald;Zhu, Haiyi
越来越多的人呼吁将受影响的社区（在线和离线）集中在设计将部署在社区中的人工智能系统中。然而，社区目标和需求的复杂性，以及人工智能开发程序、产出和潜在影响的复杂性，往往阻碍了有效的参与。在本文中，我们提出了模型卡创作工具包，该工具包支持社区成员通过审议来理解，导航和协商一系列机器学习模型，并选择最符合其集体价值观的模型。通过一系列的研讨会，我们在两个在线社区（英语和荷兰语维基百科）中对我们的方法的初始有效性进行了实证调查，并记录了我们的参与者如何集体设置基于机器学习的质量预测系统的阈值，该系统用于其社区的内容审核应用程序。我们的研究结果表明，使用模型卡创作工具包有助于提高对人工智能设计多个社区目标之间权衡的理解，让社区成员讨论和谈判权衡，并促进他们自己社区环境中的集体和知情决策。最后，我们讨论了以社区为中心的，审议驱动的人工智能设计方法所面临的挑战以及潜在的设计影响。
There have been increasing calls for centering impacted communities – both online and offline – in the design of the AI systems that will be deployed in their communities. However, the complicated nature of a community’s goals and needs, as well as the complexity of AI’s development procedures, outputs, and potential impacts, often prevents effective participation. In this paper, we present the Model Card Authoring Toolkit, a toolkit that supports community members to understand, navigate and negotiate a spectrum of machine learning models via deliberation and pick the ones that best align with their collective values. Through a series of workshops, we conduct an empirical investigation of the initial effectiveness of our approach in two online communities – English and Dutch Wikipedia, and document how our participants collectively set the threshold for a machine learning based quality prediction system used in their communities’ content moderation applications. Our results suggest that the use of the Model Card Authoring Toolkit helps improve the understanding of the trade-offs across multiple community goals on AI design, engage community members to discuss and negotiate the trade-offs, and facilitate collective and informed decision-making in their own community contexts. Finally, we discuss the challenges for a community-centered, deliberation-driven approach for AI design as well as potential design implications.
登录
查看更多内容
DOI：
10.1145/3411764.3445748
发表时间：
2021-01
期刊：
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
影响因子：
--
作者：
Samantha Robertson;Tonya Nguyen;Niloufar Salehi
通讯作者：
Samantha Robertson;Tonya Nguyen;Niloufar Salehi
影响因子：
6.5
作者：
Hsu, Yen-Chia;Huang, Ting-Hao 'kenneth';Verma, Himanshu;Mauri, Andrea;Nourbakhsh, Illah;Bozzon, Alessandro
通讯作者：
Bozzon, Alessandro
通过社区范围的审议支持虚拟团队的组建
影响因子：
--
作者：
Wen, Miaomiao;Maki, Keith;Dow, Steven;Herbsleb, James D.;Rose, Carolyn
通讯作者：
Rose, Carolyn
机器人和机器人：维基百科的免疫系统
影响因子：
2.2
作者：
Aaron L Halfaker;J. Riedl
通讯作者：
J. Riedl
DOI：
--
发表时间：
2019-05
期刊：
--
影响因子：
--
作者：
Anson Kahng;Min Kyung Lee;Ritesh Noothigattu;Ariel D. Procaccia;Alexandros Psomas
通讯作者：
Anson Kahng;Min Kyung Lee;Ritesh Noothigattu;Ariel D. Procaccia;Alexandros Psomas