Collaborative Research: The Role of Instructor and Peer Feedback in Improving the Cognitive, Interpersonal, and Intrapersonal Competencies of Student Writers in STEM Courses

促进评估方法的研究和创新（PRIME）计划旨在支持评估研究，特别强调：（1）探索确定STEM教育项目和计划的影响和有用性的创新方法;（2）建立和扩大评估STEM教育和劳动力发展计划的理论基础，包括翻译和改编其他领域的方法;以及（3）提高评估领域的能力和基础设施。该项目将通过提高学生的写作和协作技能，对科学，技术，工程和数学（STEM）教育工作者具有至关重要的意义，这些领域对经济，科学和国家安全具有重要意义。本研究的重点是在本科STEM课程的背景下，教师和同伴的互动和写作质量和改进。具体而言，该项目将通过研究教师和同伴反应对STEM课程中写作改进和知识适应的影响，绘制三个能力领域（认知，人际和自我）的发展。该项目利用了一个基于网络的评估工具，称为我的评审员（MyR）。该工具将由南佛罗里达大学（USF）、北卡罗来纳州州立大学（NCSU）、达特茅斯、马萨诸塞州理工学院（MIT）和宾夕法尼亚大学（UPenn）的大学水平生物学或化学导论的STEM教师试用。研究领域包括学业成绩和人际能力。项目交付成果将提供新的工具和程序，以帮助评估学生的知识，技能和态度，以进行项目和计划评估。大约10，000名在USF，NCSU，达特茅斯，MIT和UPenn注册STEM课程的学生将上传他们的课程写作到My Reviewers，一个评估工具，并使用该工具进行同行评审和团队项目。 这一信息是由人口统计调查和处置沿着与点击模式内的工具集补充。研究人员随后将使用学生写作能力和改进的预测建模来分析这些丰富的数据，包括基于文本的方法来识别评论，论文，同行评议，其他同行的学生评价的有用特征。评论，以及教师和学生的元反思。结果目标是（1）展示评估社区如何使用实时评估工具来创建有效的写作发展措施;（2）提供关于特定评论和评分模式对学生群体可能产生的影响的定量证据;（3）提供一个域图，以帮助STEM教育工作者更好地了解学生在STEM课程中的成功;（4）告知STEM教师同行评议的有效性。

The Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program seeks to support research on evaluation with special emphasis on: (1) exploring innovative approaches for determining the impacts and usefulness of STEM education projects and programs; (2) building on and expanding the theoretical foundations for evaluating STEM education and workforce development initiatives, including translating and adapting approaches from other fields; and (3) growing the capacity and infrastructure of the evaluation field.This project will have critical significance for Science, Technology, Engineering, and Mathematics (STEM) educators by increasing writing and collaboration skills in students, areas of importance to economics, science, and national security. This study focuses on teacher and peer interactions and writing quality and improvement in the context of undergraduate STEM courses. Specifically, the project will map the development of three competency domains (cognitive, interpersonal and intrapersonal) by researching the effects of teacher and peer response on writing improvement and knowledge adaptation in STEM courses. The project utilizes a web-based assessment tool called My Reviewers (MyR). The tool will be piloted by STEM faculty in college-level Introductory Biology or Chemistry on the campuses of University of South Florida (USF), North Carolina State University (NCSU), Dartmouth, Massachusetts Institute of Technology (MIT), and University of Pennsylvania (UPenn). Research domains include both academic performance and inter/intra-personal competencies. Project deliverables will provide new tools and procedures to assist in the assessment of students' knowledge, skills, and attitudes for project and program evaluation.Approximately 10,000 students enrolled in STEM courses at USF, NCSU, Dartmouth, MIT, and UPenn will upload their course-based writing to My Reviewers, an assessment tool, and use the tool to conduct peer reviews and team projects. This information is supplemented by surveys of demographics and dispositions along with click patterns within the toolset. Researchers will subsequently analyze this wealth of data using predictive modeling of student writing ability and improvement, including text-based methods to identify useful features of comments, papers, peer reviews, student evaluations of other peers? reviews, and instructor and student meta-reflections. Outcome goals are to (1) demonstrate ways the assessment community can use real-time assessment tools to create valid measures of writing development; (2) provide quantitative evidence regarding the likely effects of particular commenting and scoring patterns on cohorts of students; (3) offer a domain map to help STEM educators better understand student success in the STEM curriculum; and (4) inform STEM faculty regarding the efficacy of peer review.