Sub-group Fair Coding Taken to Scale for Science, Technology, Engineering, and Mathematics Learning

子组公平编码适用于科学、技术、工程和数学学习

• 批准号: 2100320
• 负责人: David Shaffer
• 金额: $ 250万
• 依托单位: University of Wisconsin-Madison
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2100320&HistoricalAwards=false
• 关键词: Sub; group; Fair; Coding; Taken

This project will advance research in an important area needed for contributing to the national need for well-educated scientists, mathematicians, engineers, and technicians through the creation and validation of a process designed to effectively and fairly code educational data. Over its five-year duration, this project will develop and test an approach to coding data on learning in science, technology, engineering, and mathematics (STEM). It will do so in a manner that takes into account differences between groups without requiring researchers to code data by hand. Thus, this proposal will save time while also being able to help with answering questions about group differences in STEM learning. This approach will be made publicly available for use by researchers coding learning data while taking into account differences across groups. As a result of these efforts, algorithms will be developed for learning science research that will: (i) produce fair classifiers, fair sets of codes, and identify conceptual shift among subgroups, (ii) provide support for identifying intersectional subgroups that may be modeled unfairly without specifying the interactions of group characteristics in advance, (iii) control for elevated Type 1 error rates, and (iv) provide an interface that can be used by researchers who care about fair coding. The technique being tested will enable this to occur with less data than is typically needed outside of learning science research. Though fairness in coding is a well-recognized challenge, particularly in STEM education research, this issue is handled almost exclusively by researchers examining their data by hand to see if the coding appears fair. This is time-consuming, and usually only done when there are strong a priori reasons to be concerned about fairness. In data science, there are relatively sophisticated approaches to fairness in coding, but there are significant limitations (e.g., the need for very large amounts of human coded data) that are currently insurmountable in learning science research. As such, having a validated method to code data, while systematically checking for subgroup fairness with the ability to identify the presence of conceptual shift will advance the field of data science in STEM learning research. By providing the protocol and algorithms associated with this process to other researchers through R Project for Statistical Computing, journal articles, and presentations will assure dissemination. This project is funded by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目将推进一个重要领域的研究，通过创建和验证一个旨在有效和公平编码教育数据的过程，为满足国家对受过良好教育的科学家、数学家、工程师和技术人员的需求做出贡献。在五年的时间里，该项目将开发和测试一种对科学、技术、工程和数学（STEM）学习数据进行编码的方法。它将以一种考虑到群体之间差异的方式进行研究，而不需要研究人员手工编码数据。因此，这一建议将节省时间，同时也能够帮助回答有关STEM学习中群体差异的问题。这种方法将公开提供给研究人员使用，以编码学习数据，同时考虑到各组之间的差异。由于这些努力，将开发用于学习科学研究的算法，这些算法将：（i）产生公平的分类器、公平的代码集，并识别子组之间的概念转变，（ii）为识别可能不公平地建模的交叉子组提供支持，而无需预先指定组特征的相互作用，（iii）控制升高的类型1错误率，以及（iv）提供可由关心公平编码的研究人员使用的接口。 正在测试的技术将使这一点能够在比学习科学研究之外通常需要的数据更少的情况下发生。虽然编码的公平性是一个公认的挑战，特别是在STEM教育研究中，但这个问题几乎完全由研究人员手工检查他们的数据来处理，看看编码是否公平。这是一项耗时的工作，而且通常只有在有充分的先验理由需要关注公平性时才这样做。在数据科学中，有相对复杂的方法来实现编码的公平性，但也有很大的局限性（例如，需要非常大量的人类编码数据），这是目前学习科学研究中无法克服的。 因此，拥有一种经过验证的方法来编码数据，同时系统地检查子组的公平性，并能够识别概念转变的存在，这将推动STEM学习研究中的数据科学领域。通过R Project for Statistical Computing向其他研究人员提供与此过程相关的协议和算法，期刊文章和演示文稿将确保传播。该项目由EHR核心研究（ECR）计划资助，该计划支持推进STEM学习和学习环境的基础研究，扩大STEM参与，以及STEM劳动力发展的工作。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

LSTM Neural Network Assisted Regex Development for Qualitative Coding

LSTM 神经网络辅助正则表达式开发进行定性编码

• 发表时间: 2022
• 期刊: International Conference on Quantitative Ethnography 2022
• 作者: Cai, Z.;Eagan, B.;Marquart C.;Shaffer, D
• 通讯作者: Shaffer, D

Quantitative Ethnography of Policy Ecosystems: A Case Study on Climate Change Adaptation Planning

政策生态系统的定量民族志：气候变化适应规划案例研究

• 发表时间: 2022
• 期刊: International Conference on Quantitative Ethnography 2022
• 作者: Ruis, A. R.

Towards strengthening links between learning analytics and assessment: Challenges and potentials of a promising new bond

加强学习分析和评估之间的联系：有希望的新纽带的挑战和潜力

• DOI: 10.1016/j.chb.2022.107304
• 发表时间: 2022
• 期刊: Computers in human behavior
• 影响因子: 9.9
• 作者: Gašević, D.;Greiff, S.;Shaffer, D. W.
• 通讯作者: Shaffer, D. W.

Neural recall network: A neural network solution to low recall problem in regex-based qualitative coding

神经召回网络：基于正则表达式的定性编码中低召回率问题的神经网络解决方案

• DOI: 10.5281/zenodo.6853047
• 发表时间: 2022
• 期刊: Proceedings of the 15th International Conference on Educational Data Mining
• 作者: Cai, Z.;Marquart, C.;Shaffer, D.
• 通讯作者: Shaffer, D.

Does Active Learning Reduce Human Coding?: A Systematic Comparison of a Neural Network with nCoder

主动学习会减少人类编码吗？：神经网络与 nCoder 的系统比较

• 发表时间: 2022
• 期刊: International Conference on Quantitative Ethnography 2022
• 作者: Choi, J.;Ruis, A. R.;Cai, Z.;Eagan, B. R.;Shaffer, D. W.
• 通讯作者: Shaffer, D. W.