The Validity of Technology-Enhanced Assessment in Geometry

评估开发人员、州教育部门和国家联盟已经将广泛的努力集中在将技术增强项目（TEIs）纳入总结性和形成性评估上。与传统的选择反应（SR）项目相比，TEIs有许多潜在的好处，包括测量更高水平构念的潜力，减少考试技巧和猜测的影响，捕获丰富的诊断信息，减少非相关构念的认知负荷，以及其设计的吸引力。前三个好处适用于构造反应（CR）项目，但tei还有一个额外的好处，即由计算机自动评分。尽管tei具有潜在的好处，并且大力推动将这些类型的项目纳入评估，但对tei所做推断的有效性以及tei是否提供了优于传统项目类型的测量方法的研究却很少。技术增强评估在几何中的有效性（VTAG）项目通过探索技术增强评估在初等几何背景下的有效性，为小型研究基地做出了贡献。该项目涉及三个研究问题：RQ1: TEIs在多大程度上是小学几何标准的有效衡量标准？与SR项目相比，tei在多大程度上提供了改进的测量？RQ3：数学标准的一般特征是什么，可以更好地通过tei来衡量？为了解决这些研究问题，研究人员为四年级和五年级几何的七个共同核心州标准中的每个标准开发了20个项目（10个SR项目和10个TEIs）。研究人员通过专家评审、认知实验室和课堂管理来收集有效性证据。前两个研究问题是通过评估基于各种来源的项目的有效性来解决的，包括测试内容，内部结构，与其他变量的关系，以及学生的反应过程。为了解决第三个研究问题，根据前两个的结果，研究人员使用定性分析来确定标准的共同主题，这些标准被认为是通过tei更好地衡量的。

Assessment developers, state departments of education, and national consortia have focused extensive efforts on including Technology-Enhanced Items (TEIs) on summative and formative assessments. TEIs have a number of potential benefits over traditional, selected-response (SR) items, including the potential to measure higher-level constructs, the reduction of the effects of test-taking skills and guessing, the capture of rich diagnostic information, the reduction of cognitive load from non-relevant constructs, and the engaging nature of their design. The first three benefits are true of constructed-response (CR) items, but TEIs have the added benefit of being automatically scored by computer. Despite the potential benefits of TEIs, and the strong push to include these types of items in assessments, there is a death of research on the validity of inferences made by TEIs and on whether TEIs provide improved measurement over traditional item types. The Validity of Technology-Enhanced Assessment in Geometry (VTAG) project contributes to the small research base by exploring the validity of TEIs in the context of elementary geometry. The project addresses three research questions: RQ1: To what extent are TEIs a valid measurement of geometry standards in the elementary grades? RQ2: To what extent do TEIs provide an improved measurement compared to SR items? RQ3: What are the general characteristics of mathematics standards that might be better measured through TEIs?To address these research questions, the researchers develop 20 items (ten SR items and 10 TEIs) for each of the seven Common Core State Standards in fourth and fifth grade geometry. The researchers collect validity evidence using expert review, cognitive labs, and classroom administration of the items. The first two research questions are addressed by evaluating the validity of the items based on a variety of sources, including test content, internal structure, the relationship to other variables, and student response processes. To address the third research question, informed by the results of the prior two, the researchers use qualitative analysis to identify common themes of the standards that were identified as being better measured through TEIs.