Collaborative Research: The Role of Instructor and Peer Feedback in Improving the Cognitive, Interpersonal, and Intrapersonal Competencies of Student Writers in STEM Courses

促进评估方法研究和创新(PRIME)计划旨在支持评估研究，特别强调：(1)探索确定STEM教育项目和计划的影响和有用性的创新方法；(2)建立和扩展评估STEM教育和劳动力发展倡议的理论基础，包括翻译和调整其他领域的方法；以及(3)增加评估领域的能力和基础设施。该项目将通过提高学生的写作和协作能力，对科学、技术、工程和数学(STEM)教育工作者具有关键意义，这些领域对经济、科学和国家安全具有重要意义。本研究以本科生STEM课程为背景，探讨教师与同学之间的互动与写作质量与提高。具体地说，该项目将通过研究教师和同伴反应对STEM课程中写作改进和知识适应的影响来描绘三个能力领域(认知、人际和个人内部)的发展。该项目利用一种名为MYR(MYR)的网络评估工具。该工具将由STEM的大学一级入门生物学或化学教师在南佛罗里达大学(USF)、北卡罗来纳州立大学(NCSU)、达特茅斯、麻省理工学院(MIT)和宾夕法尼亚大学(UPenn)的校园进行试验。研究领域既包括学业成绩，也包括人际/个人能力。项目交付成果将提供新的工具和程序来帮助评估学生在项目和项目评估中的知识、技能和态度。在USF、NCSU、达特茅斯、麻省理工学院和UPenn注册STEM课程的大约10,000名学生将把他们基于课程的写作上传到我的评审员评估工具，并使用该工具进行同行评议和团队项目。这一信息由对人口统计和处置的调查以及工具集中的点击模式补充。研究人员随后将使用学生写作能力和进步的预测性建模来分析这些丰富的数据，包括基于文本的方法来识别评论、论文、同行评议、学生对其他同行的评价的有用特征？回顾，以及教师和学生的元反思。结果目标是(1)展示评估社区如何使用实时评估工具来创建有效的写作发展措施；(2)提供关于特定评论和评分模式对学生队列的可能影响的量化证据；(3)提供领域地图以帮助STEM教育工作者更好地了解学生在STEM课程中的成功；以及(4)向STEM教职员工告知同行评议的有效性。

The Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program seeks to support research on evaluation with special emphasis on: (1) exploring innovative approaches for determining the impacts and usefulness of STEM education projects and programs; (2) building on and expanding the theoretical foundations for evaluating STEM education and workforce development initiatives, including translating and adapting approaches from other fields; and (3) growing the capacity and infrastructure of the evaluation field.This project will have critical significance for Science, Technology, Engineering, and Mathematics (STEM) educators by increasing writing and collaboration skills in students, areas of importance to economics, science, and national security. This study focuses on teacher and peer interactions and writing quality and improvement in the context of undergraduate STEM courses. Specifically, the project will map the development of three competency domains (cognitive, interpersonal and intrapersonal) by researching the effects of teacher and peer response on writing improvement and knowledge adaptation in STEM courses. The project utilizes a web-based assessment tool called My Reviewers (MyR). The tool will be piloted by STEM faculty in college-level Introductory Biology or Chemistry on the campuses of University of South Florida (USF), North Carolina State University (NCSU), Dartmouth, Massachusetts Institute of Technology (MIT), and University of Pennsylvania (UPenn). Research domains include both academic performance and inter/intra-personal competencies. Project deliverables will provide new tools and procedures to assist in the assessment of students' knowledge, skills, and attitudes for project and program evaluation.Approximately 10,000 students enrolled in STEM courses at USF, NCSU, Dartmouth, MIT, and UPenn will upload their course-based writing to My Reviewers, an assessment tool, and use the tool to conduct peer reviews and team projects. This information is supplemented by surveys of demographics and dispositions along with click patterns within the toolset. Researchers will subsequently analyze this wealth of data using predictive modeling of student writing ability and improvement, including text-based methods to identify useful features of comments, papers, peer reviews, student evaluations of other peers? reviews, and instructor and student meta-reflections. Outcome goals are to (1) demonstrate ways the assessment community can use real-time assessment tools to create valid measures of writing development; (2) provide quantitative evidence regarding the likely effects of particular commenting and scoring patterns on cohorts of students; (3) offer a domain map to help STEM educators better understand student success in the STEM curriculum; and (4) inform STEM faculty regarding the efficacy of peer review.