Sub-group Fair Coding Taken to Scale for Science, Technology, Engineering, and Mathematics Learning

该项目将通过创建和验证一个有效和公平地编码教育数据的程序，促进一个重要领域的研究，以满足国家对受过良好教育的科学家、数学家、工程师和技术人员的需求。在为期五年的时间里，该项目将开发和测试一种对科学、技术、工程和数学(STEM)学习数据进行编码的方法。它将以一种考虑到不同群体之间的差异的方式来完成这项工作，而不需要研究人员手动编码数据。因此，这项建议将节省时间，同时还能够帮助回答有关STEM学习中的群体差异的问题。这一方法将公之于众，供编码学习数据的研究人员使用，同时考虑到不同群体的差异。作为这些努力的结果，将开发用于学习科学研究的算法，这些算法将：(I)产生公平的分类器、公平的代码集，并识别子组之间的概念转移；(Ii)为识别可能被不公平地建模的交叉子组提供支持，而不预先指定组特征的相互作用；(Iii)控制提高的类型1错误率；以及(Iv)提供可由关心公平编码的研究人员使用的接口。正在测试的技术将使这种情况发生时，使用的数据比学习科学研究之外通常需要的数据更少。虽然编码的公平性是一个公认的挑战，特别是在STEM教育研究中，但这个问题几乎完全由研究人员手工检查他们的数据来处理，以确定编码是否公平。这很耗时，而且通常只有在有充分的先验理由关心公平的情况下才会这样做。在数据科学中，有相对复杂的编码公平性方法，但存在重大限制(例如，需要非常大量的人类编码数据)，这些限制目前在学习科学研究中是无法克服的。因此，拥有一种有效的编码数据的方法，同时系统地检查子组公平性，并能够识别概念迁移的存在，将推动STEM学习研究中的数据科学领域。通过R统计计算项目向其他研究人员提供与这一过程相关的协议和算法，期刊论文和演示文稿将确保传播。该项目由EHR核心研究(ECR)计划资助，该计划支持推进STEM学习和学习环境的基础研究、扩大对STEM的参与以及STEM劳动力发展的工作。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project will advance research in an important area needed for contributing to the national need for well-educated scientists, mathematicians, engineers, and technicians through the creation and validation of a process designed to effectively and fairly code educational data. Over its five-year duration, this project will develop and test an approach to coding data on learning in science, technology, engineering, and mathematics (STEM). It will do so in a manner that takes into account differences between groups without requiring researchers to code data by hand. Thus, this proposal will save time while also being able to help with answering questions about group differences in STEM learning. This approach will be made publicly available for use by researchers coding learning data while taking into account differences across groups. As a result of these efforts, algorithms will be developed for learning science research that will: (i) produce fair classifiers, fair sets of codes, and identify conceptual shift among subgroups, (ii) provide support for identifying intersectional subgroups that may be modeled unfairly without specifying the interactions of group characteristics in advance, (iii) control for elevated Type 1 error rates, and (iv) provide an interface that can be used by researchers who care about fair coding. The technique being tested will enable this to occur with less data than is typically needed outside of learning science research. Though fairness in coding is a well-recognized challenge, particularly in STEM education research, this issue is handled almost exclusively by researchers examining their data by hand to see if the coding appears fair. This is time-consuming, and usually only done when there are strong a priori reasons to be concerned about fairness. In data science, there are relatively sophisticated approaches to fairness in coding, but there are significant limitations (e.g., the need for very large amounts of human coded data) that are currently insurmountable in learning science research. As such, having a validated method to code data, while systematically checking for subgroup fairness with the ability to identify the presence of conceptual shift will advance the field of data science in STEM learning research. By providing the protocol and algorithms associated with this process to other researchers through R Project for Statistical Computing, journal articles, and presentations will assure dissemination. This project is funded by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.