CLASS: Continuous Learning and Automated Scoring in Science

这个为期五年的项目研究如何提供持续的评估和反馈，以引导学生在科学探究学习体验中的理解，以及通过技术增强系统对教师和管理人员的详细指导。该评估系统集成了针对学生对开放式评估项目的书面回答的经验证的自动评分（即，短文、科学叙述、概念图、图形化问题和虚拟实验）纳入“基于网络的探究科学环境”（WISE）计划。WISE是一个在线科学探究课程，通过对不可直接观察的过程的可视化，虚拟实验，图形化结果，协作以及对解释提示的响应来支持深入理解。该项目与教育考试服务中心合作，目标是：（1）开发五种自动化探究评估活动，以掌握学生整合思想和形成连贯科学论点的能力;（2）通过纳入自动化分数，定制WISE;（3）调查学生基于这些分数的系统反馈如何改善他们的学习成果;（4）设计专业发展资源，帮助教师利用分数改善课堂教学，帮助管理者更好地做出有关教师专业发展和探究教学的决策。该项目的目标是北方加州三个学区的普通科学（生命、物理和地球），五所中学为4,000多名具有不同文化和语言背景的6 - 8年级学生和29名科学教师提供服务。它有助于增加学生提高科学成就的机会，并为教师和管理人员做出有效的，基于证据的高质量教学和学习决策。一个关键的研究问题指导这项工作：如何自动评分的调查评估可以增加不同的学生的成功，提高教师的教学实践，并告知管理人员的专业发展，探究教学和评估的决定？为了开展科学探究评估活动，对书面回答进行评分，包括语义、句法和意义结构分析，以及通过c-rater（ETS开发的网络学习技术）使用计算机评分系统对人工评分项目进行校准。进行有效性研究，以比较自动评分与人类评分项目，教师，地区和国家的分数，包括不同的学生群体的敏感性。为了定制WISE课程，该项目修改了12个现有单元，并开发了9个新模块。为了设计自适应反馈给学生，比较研究探索自适应指导和测试替代方案的选择，基于自动评分，采用线性模型比较学生在随机分配的指导条件下的表现;控制协变量，如以前的科学成绩，性别和语言;和分组比较研究。为了设计教师专业发展，自动评分数据的综合报告，使他们能够使用证据来指导课程决策，并评论分析，以提高反馈质量。通过工作坊、课堂观察和访谈来衡量教师随时间的变化。为了授权管理员的决策，进行了特殊的数据报告，使用证据活动，个人访谈和观察管理员会议。一个负责项目评估的咨询委员会解决了形成性和总结性方面的问题。通过网络评估改善中学科学教学和学习的研究知情模式是这项努力的主要成果。共有21个新的，为期一至三周的标准为基础的科学单位，每个有四个或更多的自动评分项目，作为原型，以提高学生的表现，教师的教学方法，和管理人员的学校政策和做法。

This five-year project investigates how to provide continuous assessment and feedback to guide students' understanding during science inquiry-learning experiences, as well as detailed guidance to teachers and administrators through a technology-enhanced system. The assessment system integrates validated automated scorings for students' written responses to open-ended assessment items (i.e., short essays, science narratives, concept mapping, graphing problems, and virtual experiments) into the "Web-based Inquiry Science Environment" (WISE) program. WISE is an online science-inquiry curricula that supports deep understanding through visualization of processes not directly observable, virtual experiments, graphing results, collaboration, and response to prompts for explanations. In partnership with Educational Testing Services (ETS), project goals are: (1) to develop five automated inquiry assessment activities that capture students' abilities to integrate their ideas and form coherent scientific arguments; (2) to customize WISE by incorporating automated scores; (3) to investigate how students' systematic feedback based on these scores improve their learning outcomes; and (4) to design professional development resources to help teachers use scores to improve classroom instruction, and administrators to make better informed decisions about teacher professional development and inquiry instruction. The project targets general science (life, physical, and earth) in three northern California school districts, five middle schools serving over 4,000 6th-8th grade students with diverse cultural and linguistic backgrounds, and 29 science teachers. It contributes to increase opportunities for students to improve their science achievement, and for teachers and administrators to make efficient, evidence-based decisions about high-quality teaching and learning. A key research question guides this effort: How automated scoring of inquiry assessments can increase success for diverse students, improve teachers' instructional practices, and inform administrators' decisions about professional development, inquiry instruction, and assessment? To develop science inquiry assessment activities, scoring written responses include semantic, syntax, and structure of meaning analyses, as well as calibration of human-scored items with a computer-scoring system through the c-rater--an ETS-developed cyber learning technology. Validity studies are conducted to compare automated scores with human-scored items, teacher, district, and state scores, including sensitivity to the diverse student population. To customize the WISE curriculum, the project modifies 12 existing units and develops nine new modules. To design adaptive feedback to students, comparative studies explore options for adaptive guidance and test alternatives based on automated scores employing linear models to compare student performance across randomly assigned guidance conditions; controlling for covariates, such as prior science scores, gender, and language; and grouping comparison studies. To design teacher professional development, synthesis reports on auto-scored data are created to enable them to use evidence to guide curricular decisions, and comments' analysis to improve feedback quality. Workshops, classroom observations, and interviews are conducted to measure longitudinal teachers' change over time. To empower administrators' decision making, special data reports, using-evidence activities, individual interviews, and observation of administrators' meetings are conducted. An advisory board charged with project evaluation addresses both formative and summative aspects.A research-informed model to improve science teaching and learning at the middle school level through cyber-enabled assessment is the main outcome of this effort. A total of 21 new, one- to three-week duration standards-based science units, each with four or more automatically scored items, serve as prototypes to improve students' performance, teachers' instructional approaches, and administrators' school policies and practices.