SBIR Phase II: Applying Semantic Paradata to Outcomes-aligned Assessment

SBIR第二阶段项目将产生一个基于云的网络应用程序，该应用程序使用自然语言处理为任何数字文本材料自动生成在线评估问题，并将问题与主题和学习成果相结合。这些问题将供教师审阅、编辑，并用于学生可以在台式机、笔记本电脑、平板电脑或智能手机上进行的在线测验。该应用程序将对测验进行评分，并分析个人和总体结果。教师可以通过将错误与常见的学生误解联系起来来加强分析。该应用程序将为教师提供近乎实时的学生学习更新，并为学生提供个性化和即时的反馈。该应用程序将维护一个问题库，可以为学术单位或机构定制，并随着使用而增长。系统收集的题库和数据将被挖掘，以随着时间的推移提高性能。在第二阶段，beta版本将在多所大学进行评估。这项工作的更广泛/商业影响包括它有可能改善2100多万美国大学生的教学。评估在高等教育中发挥着至关重要的作用，但目前教师需要花费大量的时间和精力来编制、评分和维护这些评估。使用在第二阶段SBIR中开发的应用程序，教师将能够快速生成和分发测验，以便在课堂上特别使用，并且足够容易地将频繁的练习和诊断测试集成到日常教学中。这将使教师能够更好地评估学生，保持学生的参与，并根据学生的需求定制他们的教学。学生将能够参加更多的实践测试，有证据表明，这有助于记住和回忆课堂内容，管理人员将获得更多与学生成绩直接相关的数据。预期的好处包括提高学生的成功率和丰富的数据，大学和认证机构可以分析评估和改进项目。此外，第二阶段的SBIR将实现、测试和改进可用于其他应用程序的自动化问题生成和结果对齐技术。

This SBIR Phase II project will result in a cloud-based web application that uses natural language processing to automatically generate online assessment questions for any digital textual material and to align questions with topics and learning outcomes. These questions will be available for instructors to review, edit and use in online quizzes that students can take on desktops, laptops, tablets or smartphones. The application will grade the quizzes and analyze both individual and aggregate results. Instructors can enhance the analysis by tying errors to common student misconceptions. The application will provide instructors with near real-time updates on student learning and provide students with personalized and immediate feedback. The application will maintain a bank of questions that can be customized to an academic unit or institution and that will grow with use. The question bank and data collected by the system will be mined to improve performance over time. A beta release will be evaluated at multiple universities during Phase II. The broader/commercial impact of this work includes its potential to improve instruction for over 21 million American college and university students. Assessments play critical roles in higher education, but it currently takes considerable instructor time and effort to produce, grade and maintain them. Using the application developed in this Phase II SBIR, instructors will be able to generate and deliver quizzes quickly enough for ad-hoc in-class use and easily enough to integrate frequent practice and diagnostic tests into daily instruction. This will enable instructors to better evaluate students, keep students engaged, and tailor their instruction to student needs. Students will be able to take more practice tests, which evidence shows helps retain and recall class content, and administrators will obtain more data tied directly to student outcomes. Expected benefits include increased student success rates and richer data that universities and accreditation agencies can analyze to evaluate and improve programs. In addition, this Phase II SBIR will implement, test and improve techniques for automated question generation and outcomes alignment that can be used in other applications.