EAGER: Collaborative Research: Computer-Aided Response-to-Intervention for Reading Comprehension Disabilities

由44个州和哥伦比亚特区采用的共同核心学习标准（CCLS）定义了学生在每个年级结束时应该展示的技能。CCLS强调的一个关键技能是阅读能力，这是学习所有内容领域的先决条件。在纽约州，3-8年级的学生每年春天都要参加英语语言艺术（ELA）测试，以衡量他们在CCLS阅读方面的成绩。ELA测试包括多项选择题和基于短文段的开放式问题；要想做得好，学生应该能够仔细阅读一篇文章，寻找文本证据，并从中得出逻辑推论。为了报告结果，正确回答的学生数量被转换成量表分数；这又被分为四个表现水平：NYS 1级为远低于精通，NYS 2级为部分精通，NYS 3级为精通，NYS 4级为特殊的年级水平标准。学校为成绩达到NYS 1级或NYS 2级的学生安排学业干预服务。为了推动那些有可能达不到学业期望的学生做出改变，“响应干预”模式旨在根据这些评估结果提供指导。但是，个人导师认为，单一的表现分数作为评估结果往往不足以识别潜在的学习问题，特别是在阅读理解方面。在这个探索性项目中，他们将专注于发现词汇水平评估结果中的错误模式，期望这些最终将导致对如何从表现不佳的基于文本的分析阅读评估中获得的原始数据转化为信息丰富且可理解的结构的理解，从而提供有效的阅读理解干预措施。项目成果将通过支持个人层面的诊断，以及促进将具有相似阅读障碍的学生分组在同一干预组中，以优化学校教学资源，从而补充当前的评分系统。该方法还将通过最大限度地从每次考试中获得信息来支持数据驱动的教学框架，从而减少考试次数，增加每学年的教学时间。这是计算机科学家（Tsai）和扫盲教育专家（Zakierski）之间的跨学科合作。蔡皮将负责计算机算法开发和数据分析，而扎基尔斯基将负责数据收集和评估基于读写能力和教学法的建议方法。该小组将建立一个数据库，其中包含3年级以下儿童文学作品中具有词汇特性的单词、纽约公共图书馆ELAs的评估材料和干预记录。他们每年将从大约120名成绩在NYS 2级或以下的三年级学生中收集ELA评估材料，用于研究开发和评估。他们将开发一个计算机辅助干预系统，对表现不佳的个别ELA评估材料进行数据挖掘，以发现错误模式，这将有助于教师识别学生潜在的阅读理解问题，以便制定更有效的教学计划。他们将通过形成性评估和总结性评估来评估学生的表现，前者包括对教师的问卷调查和在干预期间对学生进行的模拟ELA测试，后者是在干预后的4月进行的真正的ELA测试。从计算机科学的角度来看，主要的挑战是数据集的规模太小。pi将开发领域知识驱动的新技术，用于执行有意义的分析，以发现这种情况下的错误模式；如果成功，这种方法将为其他更容易获得“小数据”的情况下的广泛研究机会打开大门。此外，对扫盲教育的数据挖掘本身的探索将构成独特的贡献，因为这两个领域的结合尚未受到研究界的太多关注，并且有许多有趣的问题等待使用计算方法来解决。

The Common Core Learning Standards (CCLS), adopted by 44 states and the District of Columbia, define the skills a student should demonstrate by the end of each grade. One key skill emphasized by CCLS is reading ability, which is the precursor for learning in all content areas. In New York State, students in grades 3-8 take an English Language Arts (ELA) test each spring to measure their CCLS achievement in reading. An ELA test contains both multiple choice questions and open-ended questions based on short text passages; to do well, students should be able to read a passage closely for textual evidence and draw logical inferences from it. To report the results, the number of correct student responses is converted into a scale score; this in turn is divided into four performance levels: NYS Level 1 for well below proficient, NYS Level 2 for partially proficient, NYS Level 3 for proficient, and NYS Level 4 for exceptional in grade-level standards. Schools arrange academic intervention services for students whose performance level is either NYS Level 1 or NYS Level 2. To drive change in students who are at risk for not meeting academic expectations, the Response-to-Intervention model aims to deliver instructions as a function of these assessment outcomes. But the PIs argue that a single performance score as the assessment outcome is often insufficient for identifying underlying learning problems, especially in reading comprehension. In this exploratory project they will focus on discovering error patterns in assessment outcomes at the lexical level, in the expectation these will ultimately lead to improved understanding of how the raw data from a pool of underperforming text-based analytic reading assessments can be transformed into an informative and understandable structure for delivery of effective reading comprehension interventions. Project outcomes will complement the current scoring system by supporting diagnosis at an individual level, and by facilitating grouping of students with similar reading disabilities in the same intervention group in order to optimize school teaching resources. The approach will also support a data-driven instruction framework by maximizing the information gain from each test, which can result in fewer tests taken and more hours for teaching per school year.This is an interdisciplinary collaboration between a computer scientist (Tsai) and an expert in literacy education (Zakierski). PI Tsai will be responsible for computer algorithm development and data analysis, whereas PI Zakierski will be in charge of data collection and evaluation of the proposed approach based on findings in literacy and pedagogy. The team will build a database containing words with lexical properties from literature for children up to grade 3, assessment materials from NYS ELAs, and intervention records. They will annually collect ELA assessment materials from a pool of approximately 120 third grade students with performance at NYS Level 2 or below, for both research development and evaluation. They will develop a computer-aided intervention system that performs data-mining on underperforming individual ELA assessment materials to discover error patterns, which should assist the teacher in identifying a student's underlying reading comprehension problems in order to prepare a more effective instruction plan. And they will evaluate the performance by doing both formative and summative assessments, the former to consist of questionnaires for teachers and mock ELA tests for students taken during the period of intervention, and the latter being the real ELA tests in April following the intervention. From the computer science perspective, the main challenge is the small size of the dataset. The PIs will develop new techniques that are domain-knowledge driven for performing meaningful analysis to discover error patterns in such situations; if successful, the approach will open the door to broad research opportunities in other cases where "small data" is easier to come by. In addition, the exploration of data mining on literacy education itself will constitute a unique contribution, since the marriage of the two fields has not yet received much attention from the research community and there are many interesting questions waiting to be addressed using computational approaches.