A Machine Learning Approach to Improve Students’ Scientific Reasoning and Writing

该项目旨在通过提高通识教育STEM课程本科生的科学推理技能来服务于国家利益。具体地说，该项目专注于帮助学生学习识别科学论点并使用基于证据的(科学)推理。入门课程通常是非理科学生接触科学的最后一次正式接触。因此，通识教育STEM课程在提高公民科学素养方面具有重要作用。为了支持这一目标，该项目将创建一个写作仪表板，使用机器学习算法来对学生的书面回答与科学证据支持其声明的程度进行评分。该项目还将开发一个网络浏览器扩展，训练学生判断互联网上的文章是否提供了支持科学主张的证据。一旦在这个探索性项目中开发了仪表板和Web浏览器扩展，机器学习工具就可以得到改进，并在全国范围内部署，供本科生和教师使用。这些工具有可能对本科教育产生重大影响，因为它们可以帮助教师评估和提供写作方面的反馈，即使在大班教学中也是如此。工具可以自动化的过程，甚至部分，可以加强书面作业和评估在STEM课堂上的使用，从而帮助学生提高他们的推理和书面沟通能力。本项目将在三个大型科学入门课程中实施和研究写作仪表板和浏览器扩展的效果。仪表板将识别代表索赔的短语对和支持这些索赔的证据。它还将根据行话的使用和可读性为写作评分。仪表板将被设计为教师使用的形成性评估工具，可以对学生的写作提供建设性的反馈。它将补充讲师的评分过程，提供一个工具来讨论与好的科学写作相关的属性。网络浏览器扩展将帮助学生在互联网上识别基于证据的科学主张。使用与写作仪表板相同的机器学习技术，浏览器扩展将识别并突出在线文章中的主张和证据，并对文章可能的科学质量给出总体评级，以及评级的理由。这些工具将在非科学专业的三门入门科学课程中学习：天文学、地球科学和进化生物学。学生将被要求使用仪表板完成三项写作作业，他们将使用浏览器扩展程序进行要求他们审查和评分在线科学文章的活动。为了研究学生科学推理能力的潜在改善，该项目将调整现有的调查工具，并在干预前后对学生进行修订后的调查。我们将与教师进行访谈，以了解这些工具在课堂上的用途。除了大学环境，这些工具也可以在高中使用，浏览器扩展可以部署在图书馆和其他非正式环境中，以帮助提高普通人群的科学素养和推理能力。该项目得到了NSF改善本科生STEM教育计划：教育和人力资源计划的支持，该计划支持研究和开发项目，以提高所有学生的STEM教育效率。通过参与的学生学习路径，该计划支持有前景的实践和工具的创建、探索和实施。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project aims to serve the national interest by improving the scientific reasoning skills of undergraduates in general education STEM courses. Specifically, the project focuses on helping students learn to recognize scientific arguments and use evidence-based (scientific) reasoning. Introductory courses are typically the last formal exposure to science that non-science students will have. Thus, general education STEM courses have a significant role in increasing civic science literacy. To support this goal, the project will create a writing dashboard that uses a machine learning algorithm to score how well a student’s written response supports its claims with scientific evidence. The project will also develop a web browser extension that trains students to determine whether articles on the internet provide evidence to support scientific claims. Once the dashboard and web browser extension are developed in this exploratory project, the machine learning tools can be improved and deployed nationally for use by undergraduate students and instructors. These tools have the potential for significant impact on undergraduate education, since they can assist instructors with assessing and providing feedback on writing, even in large classes. Tools that can automate the process, even partially, could enhance the use of written assignments and assessments in STEM classes, thus helping students increase their reasoning and written communication skills.This project will implement and study the efficacy of a writing dashboard and browser extension in three large introductory science courses. The dashboard will identify pairs of phrases that represent claims and evidence to support those claims. It will also score writing based on its use of jargon and its readability. The dashboard will be designed for instructors to use as a formative assessment tool that can provide constructive feedback on student writing. It will complement the instructor’s grading process, providing a vehicle for discussing attributes associated with good scientific writing. The web browser extension will help students identify evidence-based scientific claims on the internet. Using the same machine learning technology as the writing dashboard, the browser extension will identify and highlight claims and evidence in articles available online and give an overall rating for the article’s likely scientific quality, along with a rationale for the rating. The tools will be studied in three introductory science courses taken by non-science majors: astronomy, geosciences, and evolutionary biology. Students will be required to use the dashboard for three writing assignments, and they will use the browser extension for activities that require them to review and rate online scientific articles. To study potential improvements in students’ scientific reasoning capacity, the project will adapt existing survey instruments and administer the revised surveys to students before and after the intervention. Instructors will be interviewed to understand the utility of the tools in the classroom. Beyond the university setting, these tools can also be used in high schools and the browser extension can be deployed in libraries and other informal settings to help improve scientific literacy and reasoning skills within the general population. This project is supported by the NSF Improving Undergraduate STEM Education Program: Education and Human Resources Program, which supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.