Collaborative Research: Developing & Evaluating Assessments of Problem Solving

目前的国家数学标准在内容标准和实践标准上都侧重于解决问题的能力。内容标准描述了每个年级的数学学生应该学习什么，而实践标准描述了所有学生都应该经历的数学行为（例如，在解决问题和对现实世界情况进行有效推理时的毅力）。每个年级都有解决问题的能力。如果期望数学教师在日常教学中让学生参与解决问题，那么学生解决问题的表现必须以产生有意义、有效和可靠的分数的方式进行评估，而不会给教师或学生带来过度的负担。不幸的是，大多数解决问题的评估通常是由一套不同于国家标准的数学期望所框定的。因此，这些评估的结果与学生在课堂上学到的数学内容是脱节的。在此之前，本研究小组已经为6-8年级建立了解决问题的措施，解决了框架中的这一差距，并产生了有意义、有效和可靠的分数，并且不会对学生产生意想不到的负面影响。目前的项目名为“开发和评估解决问题的评估”（DEAP），该项目以该团队之前的工作为基础，为3-5年级的学生制定了解决问题的措施。小学阶段的评估将与中学阶段的评估相结合，供学区、研究人员和其他教育专业人员使用，以有效地衡量儿童解决问题的能力。探索研究K-12项目（DRK-12）旨在通过研究和开发创新资源、模型和工具（rmt），显著提高pre -12学生和教师对科学、技术、工程和数学（STEM）的学习和教学。DRK-12计划中的项目建立在STEM教育的基础研究和先前的研究和开发工作的基础上，为拟议的项目提供了理论和实证依据。从广义上讲，DEAP的目标是(a)创建三种新的数学问题解决评估并收集其使用的有效性证据，(b)将问题解决措施（psm）与先前的问题解决措施（即PSM6， PSM7和PSM8）联系起来，以及(c)为psm开发一个有意义的报告系统。研究问题是：(a) PSM3、PSM4和PSM5的心理测量特性与学生的问题解决表现有什么关系？(b)证据如何支持PSM3、PSM4、PSM5、PSM6、PSM7和PSM8的垂直相等（联系）？(c) PSM3、PSM4和PSM5及其相关报告系统在正式使用时如何影响教师的教学决策？第一年的重点是项目和测试开发。该研究将对一小群学生进行认知访谈和管理测试，以探索项目和测试的功能。将采用Rasch （1-PL）测量，类似于先前的PSM开发。第二年包括进一步的试点测试，并通过认知访谈和测试管理收集有效性证据。三年级有最后一轮试点测试和选择垂直等式的链接项目。第四年涉及PSM系列与链接项目的试点测试和开发一个报告系统。DEAP对该领域的潜在贡献有三个方面。(1)评估将供公众使用。(2)一套垂直相等的问题解决措施将允许用户有机会探索学生在跨年级入学时解决问题的表现，这目前在州或国家层面是不可能的。(3)该项目填补了该领域的需求，因为在州测试的背景下，没有一套措施使用垂直等同来严格评估小学生解决问题的表现。

Current state standards in mathematics are strategically focused on problem-solving skills in both content standards and practice standards. Content standards describe what math students are expected to learn at each grade level while practice standards characterize math behaviors that all students should experience (e.g., perseverance while problem solving and reasoning effectively about real-world situations). Problem solving is found at every grade level. If math teachers are expected to engage students in problem solving during everyday instruction, then students' problem-solving performance must be assessed in a manner that produces meaningful, valid, and reliable scores, without unduly burdening teachers or students. Unfortunately, most problem-solving assessments are generally framed by a set of mathematics expectations that differ from state standards. Thus, results from those assessments are disconnected from the mathematics content that students learn in the classroom. Previously, this research team has built problem-solving measures for grades 6-8, which address this gap in framing and generates meaningful, valid, and reliable scores, and do not have unintended negative consequences on students. The current project, titled Developing and Evaluating Assessments of Problem Solving (DEAP), builds upon the team's prior work by creating problem-solving measures for grades 3-5. The elementary assessments will be connected to the middle-grades assessments and will be available for use by school districts, researchers, and other education professionals seeking to effectively measure children's problem solving. The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools (RMTs). Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects.Broadly speaking, the aims of DEAP are (a) to create three new mathematical problem-solving assessments and gather validity evidence for their use, (b) link the problem-solving measures (PSMs) with prior problem-solving measures (i.e., PSM6, PSM7, and PSM8), and (c) develop a meaningful reporting system for the PSMs. The research questions are: (a) What are the psychometric properties of the PSM3, PSM4, and PSM5 as they relate to students' problem-solving performance? (b) How does the evidence support vertical equating (linking) of the PSM3, PSM4, PSM5, PSM6, PSM7, and PSM8? (c) How do the PSM3, PSM4, and PSM5, and their related reporting systems impact teachers' instructional decision making when used formatively? Year 1 focuses on item and test development. The study will conduct cognitive interviews and administer tests with a small group of students to explore how items and tests function. Rasch (1-PL) measurement will be employed, similar to prior PSM development. Year 2 includes further pilot testing and gathering validity evidence through cognitive interviews and test administration. Year 3 has a final round of pilot testing and selection of linking items for vertical equating. Year 4 involves pilot testing the PSM series with linking items and developing a reporting system. DEAP's potential contributions to the field are three-fold. (1) Assessments will be available for use by the public. (2) A set of vertically equated problem-solving measures will allow users the opportunity to explore students' problem-solving performance as they matriculate across grade levels, which is currently not possible at the state or national level. (3) This project fills a need in the field as no set of measures uses vertical equating to assess elementary students' problem-solving performance in a rigorous fashion within the context of state testing.