CLASS: Continuous Learning and Automated Scoring in Science

这个为期五年的项目研究如何提供持续的评估和反馈，以指导学生在科学探究学习过程中的理解，并通过技术增强的系统为教师和管理人员提供详细的指导。评估系统将学生对开放式评估项目（即短文、科学叙述、概念图、图表问题和虚拟实验）的书面回答的有效自动评分整合到“基于网络的探究科学环境”（WISE）计划中。WISE是一个在线科学探究课程，通过可视化的过程，不直接观察，虚拟实验，绘图结果，协作，并响应提示解释，支持深刻的理解。与美国教育考试服务中心（ETS）合作，项目目标是：(1)开发五种自动询问评估活动，以捕获学生整合其想法和形成连贯科学论据的能力；(2)通过纳入自动评分来定制WISE；(3)调查学生基于这些分数的系统反馈如何提高他们的学习成果；(4)设计专业发展资源，帮助教师利用分数改善课堂教学，帮助管理者对教师专业发展和探究性教学做出更明智的决策。该项目针对北加州三个学区的普通科学（生命、物理和地球），五所中学服务于4000多名具有不同文化和语言背景的6 -8年级学生，以及29名科学教师。它有助于增加学生提高科学成绩的机会，并有助于教师和管理人员就高质量的教与学做出有效的、基于证据的决策。一个关键的研究问题指导了这一努力：探究性评估的自动评分如何提高不同学生的成功率，改善教师的教学实践，并告知管理者关于专业发展、探究性指导和评估的决策？为了开展科学探究评估活动，书面回答的评分包括语义、句法和意义结构分析，以及通过c-rater （ets开发的网络学习技术）使用计算机评分系统对人工评分项目进行校准。进行有效性研究，将自动评分与人工评分项目、教师、地区和州的评分进行比较，包括对不同学生群体的敏感性。为了定制WISE课程，该项目修改了12个现有单元，并开发了9个新模块。为了给学生设计自适应反馈，比较研究探讨了自适应指导和基于自动评分的测试方案，采用线性模型比较随机分配的指导条件下学生的表现；控制协变量，如先前的科学成绩、性别和语言；分组比较研究。为了设计教师的专业发展，创建了自动评分数据的综合报告，使他们能够使用证据来指导课程决策，并对评论进行分析，以提高反馈质量。通过研讨会、课堂观察和访谈来衡量教师随时间的纵向变化。为了增强管理员的决策能力，进行了特殊的数据报告、使用证据活动、个人访谈和对管理员会议的观察。负责项目评价的咨询委员会处理形成性和总结性两个方面。这一努力的主要成果是建立一个以研究为基础的模型，通过网络评估来改善中学的科学教学。共有21个新的、为期一到三周的基于标准的科学单元，每个单元有四个或更多的自动评分项目，作为改进学生表现、教师教学方法和管理人员学校政策和实践的原型。

This five-year project investigates how to provide continuous assessment and feedback to guide students' understanding during science inquiry-learning experiences, as well as detailed guidance to teachers and administrators through a technology-enhanced system. The assessment system integrates validated automated scorings for students' written responses to open-ended assessment items (i.e., short essays, science narratives, concept mapping, graphing problems, and virtual experiments) into the "Web-based Inquiry Science Environment" (WISE) program. WISE is an online science-inquiry curricula that supports deep understanding through visualization of processes not directly observable, virtual experiments, graphing results, collaboration, and response to prompts for explanations. In partnership with Educational Testing Services (ETS), project goals are: (1) to develop five automated inquiry assessment activities that capture students' abilities to integrate their ideas and form coherent scientific arguments; (2) to customize WISE by incorporating automated scores; (3) to investigate how students' systematic feedback based on these scores improve their learning outcomes; and (4) to design professional development resources to help teachers use scores to improve classroom instruction, and administrators to make better informed decisions about teacher professional development and inquiry instruction. The project targets general science (life, physical, and earth) in three northern California school districts, five middle schools serving over 4,000 6th-8th grade students with diverse cultural and linguistic backgrounds, and 29 science teachers. It contributes to increase opportunities for students to improve their science achievement, and for teachers and administrators to make efficient, evidence-based decisions about high-quality teaching and learning. A key research question guides this effort: How automated scoring of inquiry assessments can increase success for diverse students, improve teachers' instructional practices, and inform administrators' decisions about professional development, inquiry instruction, and assessment? To develop science inquiry assessment activities, scoring written responses include semantic, syntax, and structure of meaning analyses, as well as calibration of human-scored items with a computer-scoring system through the c-rater--an ETS-developed cyber learning technology. Validity studies are conducted to compare automated scores with human-scored items, teacher, district, and state scores, including sensitivity to the diverse student population. To customize the WISE curriculum, the project modifies 12 existing units and develops nine new modules. To design adaptive feedback to students, comparative studies explore options for adaptive guidance and test alternatives based on automated scores employing linear models to compare student performance across randomly assigned guidance conditions; controlling for covariates, such as prior science scores, gender, and language; and grouping comparison studies. To design teacher professional development, synthesis reports on auto-scored data are created to enable them to use evidence to guide curricular decisions, and comments' analysis to improve feedback quality. Workshops, classroom observations, and interviews are conducted to measure longitudinal teachers' change over time. To empower administrators' decision making, special data reports, using-evidence activities, individual interviews, and observation of administrators' meetings are conducted. An advisory board charged with project evaluation addresses both formative and summative aspects.A research-informed model to improve science teaching and learning at the middle school level through cyber-enabled assessment is the main outcome of this effort. A total of 21 new, one- to three-week duration standards-based science units, each with four or more automatically scored items, serve as prototypes to improve students' performance, teachers' instructional approaches, and administrators' school policies and practices.