Exploring Theory and Design Principles (ETD): Auditing Machine Learning Applications for Algorithmic Justice with Computer Science High School Students and Teachers

迫切需要开发和实施学习计划，帮助教师为学生有效地与机器学习应用程序进行交互和批判性评估做好准备。该项目将与加州、特拉华州和宾夕法尼亚州的农村、郊区和城市社区的一组高中教师合作，为包括黑人、拉丁裔和性别边缘化的年轻人在内的不同高中生提供服务，设计和实施课堂活动，支持学生开发和“审核”机器学习应用程序。算法审计的目标是通过反复查询人工智能系统来更好地理解人工智能系统不透明的内部运作，以解释其外部效果和影响。该项目由学生和教师创新技术体验（ITEST）计划资助，该计划支持建立对实践，计划要素，背景和过程的理解的项目，有助于增加学生对科学，技术，工程和数学（STEM）以及信息和通信技术（ICT）职业的知识和兴趣。210名高中生将直接参与算法审计的学习。研究人员和计算机科学高中教师将共同调查学生的机器学习驱动的，基于传感器的应用程序，并整合协作算法审计实践，以检查公平性，问责制和正义。该研究将与来自探索计算机科学社区的公立高中教师合作进行，并解决以下问题：（a）高中计算机科学教师在机器学习应用中的算法正义的价值观和考虑因素是什么？(b)高中计算机科学教师如何通过审计理解和发展机器学习应用中的算法正义的新理解？教师发现算法审计对他们的教学和学生有什么影响？，(c)高中计算机科学教师如何整合和支持学生在课堂上对机器学习应用程序的协作审计？以及（d）高中生在审核机器学习应用程序时的方法和观点是什么？这如何影响他们在CS和STEM职业生涯的兴趣？为了回答这些研究问题，研究人员将使用观察方法，访谈和调查相结合的方法来深入了解教师对算法审计的理解，观察高中生设计和审计机器学习应用程序，并检查学生在课堂上实施审计活动的学习情况。以前对非专家（包括成年人和青年人）的算法审计的研究只集中在非正式环境中。在这项以学校为基础的调查中获得的关于教学和学习算法审计的见解可以扩展到许多现有的计算活动以及K-12计算课程和教师专业发展。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

There is an urgent need to develop and implement learning programs that can help teachers to prepare students to effectively interact with and critically evaluate machine learning applications. This project will work with a group of high school teachers across rural, suburban, and urban US communities in California, Delaware and Pennsylvania serving diverse high school students including Black, Latinx, and gender-marginalized young people to design and implement classroom activities that will support students in developing and “auditing” machine learning applications. The goal of algorithm auditing is to better understand the opaque inner workings of AI systems by repeatedly querying the AI system in order to interpret its external effects and impacts. This project is funded by the Innovative Technology Experiences for Students and Teachers (ITEST) program, which supports projects that build understandings of practices, program elements, contexts and processes contributing to increasing students' knowledge and interest in science, technology, engineering, and mathematics (STEM) and information and communication technology (ICT) careers.In the three-year study, 210 high school students will directly participate in the learning of algorithm auditing. Researchers and computer science high school teachers will co-investigate students’ machine-learning-powered, sensor-based applications and integrate collaborative algorithm auditing practices to examine fairness, accountability, and justice. The research will be conducted in partnership with public high school teachers from the Exploring Computer Science community and address the following questions: (a) What are high school computer science teachers’ values and considerations of algorithmic justice in machine learning applications?, (b) How do high school computer science teachers understand and develop new understandings of algorithmic justice in machine learning applications through auditing?, What implications do teachers find in algorithm auditing for their teaching and with their students?, (c) How do high school computer science teachers integrate and support students’ collaborative audits of machine learning applications in classrooms?, and (d) What are high school students’ approaches and perspectives when auditing machine learning applications? How does this affect their interests in CS and STEM careers? To answer these research questions, researchers will use a combination of observational methods, interviews and surveys to gain insights into teachers’ understandings of algorithm audits, observe high school students designing and auditing machine learning applications, and examine student learning in classroom implementation of auditing activities. Previous research on algorithm auditing for non-experts, including both adults and youth, has only focused on informal settings. The insights gained about teaching and learning algorithm auditing in this school-based investigation can be extended to and inform numerous existing computing activities as well as K–12 computing curricula and teacher professional development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.