A Machine Learning Approach to Improve Students’ Scientific Reasoning and Writing

本项目旨在通过提高通识教育STEM课程本科生的科学推理能力，为国家利益服务。具体而言，该项目侧重于帮助学生学会识别科学论点并使用基于证据的（科学）推理。导论课程通常是非理科生最后一次正式接触科学的课程。因此，通识教育STEM课程在提高公民科学素养方面具有重要作用。为了实现这一目标，该项目将创建一个写作仪表板，使用机器学习算法对学生的书面回答进行评分，以科学证据支持其主张。该项目还将开发一个网络浏览器扩展，训练学生判断互联网上的文章是否提供了支持科学主张的证据。一旦在这个探索性项目中开发出仪表板和web浏览器扩展，机器学习工具就可以得到改进，并在全国范围内部署，供本科生和教师使用。这些工具有可能对本科教育产生重大影响，因为它们可以帮助教师评估和提供写作反馈，即使在大班中也是如此。能够自动化这一过程的工具，即使是部分自动化，也可以提高STEM课程中书面作业和评估的使用，从而帮助学生提高推理和书面沟通能力。本项目将在三门大型科学入门课程中实现并研究编写仪表板和浏览器扩展的效果。仪表板将识别代表主张和支持这些主张的证据的短语对。它还会根据行话的使用和可读性来评分。仪表板将被设计为教师使用的形成性评估工具，可以对学生的写作提供建设性的反馈。它将补充教师的评分过程，为讨论与优秀科学写作相关的属性提供一个工具。网络浏览器扩展将帮助学生在互联网上识别基于证据的科学主张。使用与写作仪表板相同的机器学习技术，浏览器扩展将识别和突出显示在线文章中的主张和证据，并对文章可能的科学质量进行总体评级，以及评级的基本原理。这些工具将在非科学专业的三门入门科学课程中学习：天文学、地球科学和进化生物学。学生将被要求使用仪表板完成三个写作作业，他们将使用浏览器扩展进行需要他们审查和评价在线科学文章的活动。为了研究学生科学推理能力的潜在改善，该计划将调整现有的调查工具，并在干预前后对学生进行修订后的调查。教师将接受采访，以了解这些工具在课堂上的效用。除了大学环境，这些工具也可以在高中使用，浏览器扩展可以在图书馆和其他非正式环境中部署，以帮助提高普通人群的科学素养和推理技能。本项目由美国国家科学基金会改善本科STEM教育计划：教育和人力资源计划支持，该计划支持研究和开发项目，以提高所有学生STEM教育的有效性。通过参与学生学习轨道，该计划支持有前途的实践和工具的创建，探索和实施。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to serve the national interest by improving the scientific reasoning skills of undergraduates in general education STEM courses. Specifically, the project focuses on helping students learn to recognize scientific arguments and use evidence-based (scientific) reasoning. Introductory courses are typically the last formal exposure to science that non-science students will have. Thus, general education STEM courses have a significant role in increasing civic science literacy. To support this goal, the project will create a writing dashboard that uses a machine learning algorithm to score how well a student’s written response supports its claims with scientific evidence. The project will also develop a web browser extension that trains students to determine whether articles on the internet provide evidence to support scientific claims. Once the dashboard and web browser extension are developed in this exploratory project, the machine learning tools can be improved and deployed nationally for use by undergraduate students and instructors. These tools have the potential for significant impact on undergraduate education, since they can assist instructors with assessing and providing feedback on writing, even in large classes. Tools that can automate the process, even partially, could enhance the use of written assignments and assessments in STEM classes, thus helping students increase their reasoning and written communication skills.This project will implement and study the efficacy of a writing dashboard and browser extension in three large introductory science courses. The dashboard will identify pairs of phrases that represent claims and evidence to support those claims. It will also score writing based on its use of jargon and its readability. The dashboard will be designed for instructors to use as a formative assessment tool that can provide constructive feedback on student writing. It will complement the instructor’s grading process, providing a vehicle for discussing attributes associated with good scientific writing. The web browser extension will help students identify evidence-based scientific claims on the internet. Using the same machine learning technology as the writing dashboard, the browser extension will identify and highlight claims and evidence in articles available online and give an overall rating for the article’s likely scientific quality, along with a rationale for the rating. The tools will be studied in three introductory science courses taken by non-science majors: astronomy, geosciences, and evolutionary biology. Students will be required to use the dashboard for three writing assignments, and they will use the browser extension for activities that require them to review and rate online scientific articles. To study potential improvements in students’ scientific reasoning capacity, the project will adapt existing survey instruments and administer the revised surveys to students before and after the intervention. Instructors will be interviewed to understand the utility of the tools in the classroom. Beyond the university setting, these tools can also be used in high schools and the browser extension can be deployed in libraries and other informal settings to help improve scientific literacy and reasoning skills within the general population. This project is supported by the NSF Improving Undergraduate STEM Education Program: Education and Human Resources Program, which supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.