SBIR Phase I: Authoring tools for rapid high quality complex assessments for Next Generation Science Standards

这个SBIR第一阶段项目将研究创作的新工具的可行性；分发和评分下一代科学标准基于游戏的高质量综合表现评估，最终目标是整个K-12科学部分，近5400万美国学生。就全球竞争力而言，美国科学教育正面临严峻挑战。为了应对这些挑战，下一代科学标准要求在评估方面做出重大改变。从历史上看，评估的开发生命周期，特别是用于衡量复杂学习的评估，需要大量的时间和资源。此外，内容和认知的复杂测量的设计、开发和验证可能很昂贵，而且需要数月时间。同样，创建模拟和游戏也非常昂贵。能够创作大量高质量基于游戏的评估的软件系统将加速评估的开发和创新，降低成本，提高教师的熟练程度和接受度，并通过提供关于学生学习进度的有价值的反馈来积极影响教学和课程。高参与度的评估将潜在地帮助学生认识到他们需要在没有典型测试压力的情况下将精力集中在哪里，从而增加学习和自我效能结果，并显著增加决定攻读理工科学位和职业的学生数量。这项SBIR建议通过创建一个基于游戏的新型评估编写系统，寻求同时满足几个重要的评估要求，包括质量、开发成本、学生参与度、易于编写和教师认可。拟议的方法将调查两个备选方案之间的中间路径。传统的方法是由专业人员开发评估项目。虽然结果是高质量的，但它们也很昂贵，缺乏老师的认同和学生的参与。相比之下，另一种方法要求教师在没有太多专家指导的情况下开发和测试新试题。这些商品虽然货真价实，但也可能存在质量问题。这项拟议的研究将调查创作工具，这些工具让教师在创作真实的评估时可以自由地发挥他们的创造力，同时也提供专家指导，以显著提高评估的高质量并产生值得信赖的形成性反馈的可能性。拟议的系统还将使用心理测量学和教育数据挖掘技术自动测量和评分学生的表现，以快速产生可操作的见解，指导下一个最佳教学步骤。拟议的创作工具和由此产生的评估将通过可用性、课堂和众包研究进行验证。

This SBIR Phase 1 project will research the feasibility of novel tools for authoring; distributing and scoring high quality game based complex performance assessments for Next Generation Science Standards, ultimately targeting the entire K-12 science segment of nearly 54 million students in the US. US science education is facing critical challenges in terms of global competitiveness. Designed to meet these challenges, Next Generation Science Standards call for major changes in assessments. Historically, the development life cycle of assessments, particularly for measuring complex learning, has required significant amount of time and resources. In addition, the design, development, and validation of complex measures of content and cognition can be expensive and take many months. Similarly, creation of simulations and games is also very expensive. Software systems that enable the authoring of large numbers of high quality game based assessments will accelerate assessment development and innovation, reduce costs, enhance teacher proficiency and buy-in, and also positively impact instruction and curriculum by providing valuable feedback on student learning progression. Highly engaging assessments will potentially help students realize where they need to focus their efforts without the pressure of typical tests, thereby increasing learning and self-efficacy outcomes and significantly increasing the number of students who decide to pursue science and engineering degrees and careers.This SBIR proposal seeks to simultaneously satisfy several important assessment requirements including quality, development cost, student engagement, easy authoring and teacher buy-in by creating a novel game based assessment authoring system. The proposed approach will investigate a middle path between two alternatives. The traditional approach involves assessment item development by professionals. While the resulting items are of high quality, they are also expensive and lack teacher buy-in and student engagement. In contrast, another approach calls for teachers to develop and test new items without much expert guidance. These items, while authentic and low-cost, can have quality problems. The proposed research will investigate authoring tools that offer teachers the freedom to exercise their creativity in authoring authentic assessments, while also providing expert guidance to significantly increase the probability that the assessments are of high quality and generate trustworthy formative feedback. The proposed system will also automatically measure and score students' performances using techniques from psychometrics and educational data mining to rapidly produce actionable insights to guide the next best instructional steps. The proposed authoring tools and resulting assessments will be validated with usability, classroom and crowdsourcing studies.