Modeling Assumptions Clash with the Real World: Transparency, Equity, and Community Challenges for Student Assignment Algorithms
Modeling Assumptions Clash with the Real World: Transparency, Equity, and Community Challenges for Student Assignment Algorithms
复制标题
DOI：
10.1145/3411764.3445748
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
Samantha Robertson;Tonya Nguyen;Niloufar Salehi
中科院分区：
文献类型：
--
作者：
Samantha Robertson;Tonya Nguyen;Niloufar Salehi
在美国，越来越多的学区正在转向匹配算法来分配学生到公立学校。这些算法的设计者旨在促进过程中的透明度，公平和社区等价值。然而，学区在部署中遇到了实际挑战。事实上，旧金山弗朗西斯科联合学区投票决定停止使用并完全重新设计他们的学生分配算法，因为这对家庭来说是令人沮丧的，而且在实践中没有促进教育公平。我们分析了这个系统使用的价值敏感的设计方法，并发现，在实践中的价值观没有得到满足的一个原因是，该系统依赖于建模假设家庭的优先级，约束条件和目标，冲突与真实的世界。这些假设忽视了许多家庭面临的理想参与的复杂障碍，特别是由于社会经济不平等。我们认为，与利益相关者直接、持续的接触是使算法价值与真实的世界条件保持一致的核心。在这样做的时候，我们必须拓宽我们评估算法的方式，同时认识到纯算法解决方案在解决复杂社会政治问题方面的局限性。
Across the United States, a growing number of school districts are turning to matching algorithms to assign students to public schools. The designers of these algorithms aimed to promote values such as transparency, equity, and community in the process. However, school districts have encountered practical challenges in their deployment. In fact, San Francisco Unified School District voted to stop using and completely redesign their student assignment algorithm because it was frustrating for families and it was not promoting educational equity in practice. We analyze this system using a Value Sensitive Design approach and find that one reason values are not met in practice is that the system relies on modeling assumptions about families’ priorities, constraints, and goals that clash with the real world. These assumptions overlook the complex barriers to ideal participation that many families face, particularly because of socioeconomic inequalities. We argue that direct, ongoing engagement with stakeholders is central to aligning algorithmic values with real world conditions. In doing so we must broaden how we evaluate algorithms while recognizing the limitations of purely algorithmic solutions in addressing complex socio-political problems.