Collaborative Research: The Role of Instructor and Peer Feedback in Improving the Cognitive, Interpersonal, and Intrapersonal Competencies of Student Writers in STEM Courses

促进评估方法的研究和创新（PRIME）计划旨在支持评估研究，特别强调：(1)探索确定STEM教育项目和计划的影响和有用性的创新方法；(2)建立和扩展评估STEM教育和劳动力发展倡议的理论基础，包括翻译和适应其他领域的方法；(3)发展评价领域的能力和基础设施。该项目将通过提高学生在经济、科学和国家安全重要领域的写作和协作技能，对科学、技术、工程和数学（STEM）教育工作者具有至关重要的意义。本研究的重点是在本科STEM课程的背景下，教师和同伴的互动以及写作质量和提高。具体而言，该项目将通过研究教师和同伴的反应对STEM课程中写作改进和知识适应的影响，绘制三个能力领域（认知、人际和人际关系）的发展图。该项目利用了一种基于网络的评估工具，称为My reviewer （MyR）。该工具将由南佛罗里达大学（USF）、北卡罗来纳州立大学（NCSU）、达特茅斯大学、麻省理工学院（MIT）和宾夕法尼亚大学（UPenn）校园内大学水平入门生物学或化学的STEM教师进行试点。研究领域包括学习成绩和人际/内部能力。项目可交付成果将提供新的工具和程序，以协助评估学生的知识、技能和对项目和项目评估的态度。大约1万名就读于南加州大学、NCSU、达特茅斯、麻省理工学院和宾夕法尼亚大学STEM课程的学生将把他们的课程写作上传到My reviews，这是一个评估工具，并使用该工具进行同行评审和团队项目。这些信息是通过对人口统计和倾向的调查以及工具集中的点击模式进行补充的。研究人员随后将使用学生写作能力和进步的预测模型来分析这些丰富的数据，包括基于文本的方法来识别评论、论文、同行评审、学生对其他同行的评估的有用特征。回顾，以及教师和学生的元反思。结果目标是(1)展示评估社区如何使用实时评估工具来创建有效的写作发展措施；(2)就特定的评论和评分模式对学生群体的可能影响提供定量证据；(3)提供领域图，帮助STEM教育工作者更好地了解学生在STEM课程中的成功；(4)告知STEM教师同行评议的有效性。

The Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program seeks to support research on evaluation with special emphasis on: (1) exploring innovative approaches for determining the impacts and usefulness of STEM education projects and programs; (2) building on and expanding the theoretical foundations for evaluating STEM education and workforce development initiatives, including translating and adapting approaches from other fields; and (3) growing the capacity and infrastructure of the evaluation field.This project will have critical significance for Science, Technology, Engineering, and Mathematics (STEM) educators by increasing writing and collaboration skills in students, areas of importance to economics, science, and national security. This study focuses on teacher and peer interactions and writing quality and improvement in the context of undergraduate STEM courses. Specifically, the project will map the development of three competency domains (cognitive, interpersonal and intrapersonal) by researching the effects of teacher and peer response on writing improvement and knowledge adaptation in STEM courses. The project utilizes a web-based assessment tool called My Reviewers (MyR). The tool will be piloted by STEM faculty in college-level Introductory Biology or Chemistry on the campuses of University of South Florida (USF), North Carolina State University (NCSU), Dartmouth, Massachusetts Institute of Technology (MIT), and University of Pennsylvania (UPenn). Research domains include both academic performance and inter/intra-personal competencies. Project deliverables will provide new tools and procedures to assist in the assessment of students' knowledge, skills, and attitudes for project and program evaluation.Approximately 10,000 students enrolled in STEM courses at USF, NCSU, Dartmouth, MIT, and UPenn will upload their course-based writing to My Reviewers, an assessment tool, and use the tool to conduct peer reviews and team projects. This information is supplemented by surveys of demographics and dispositions along with click patterns within the toolset. Researchers will subsequently analyze this wealth of data using predictive modeling of student writing ability and improvement, including text-based methods to identify useful features of comments, papers, peer reviews, student evaluations of other peers? reviews, and instructor and student meta-reflections. Outcome goals are to (1) demonstrate ways the assessment community can use real-time assessment tools to create valid measures of writing development; (2) provide quantitative evidence regarding the likely effects of particular commenting and scoring patterns on cohorts of students; (3) offer a domain map to help STEM educators better understand student success in the STEM curriculum; and (4) inform STEM faculty regarding the efficacy of peer review.