Real-Time Multi-Dimensional Assessment of Student Learning

正在开发一种基于网络的评估工具，为学生的学习提供实时、多层面、形成性的评估。这个工具提供了根据主题和掌握程度来衡量学习的方法。掌握水平被定义为信息水平、算法水平、概念水平和问题解决水平。信息水平的特点是记忆和回忆、重复信息片段以及识别相关信息的能力。算法层面的特点是能够模仿、执行指令，并在熟悉的背景下使用记忆中的信息。概念层面的特点是能够可视化、重新表述、更改表示法、建立联系和提供解释。问题解决水平的特点是在新的背景下使用材料的能力；分析问题的能力；识别解决问题所需的信息、算法和理解的能力；将这些组成部分综合为解决方案的能力；以及评估该解决方案的质量的能力。该项目产生了四个产品：1)建立在问题编码数据库基础上的网络评估系统；2)编码问题数据库，用于衡量学生在大学化学入门课程中的学习情况，经校准以复制通过学生访谈产生的测量结果；3)开发和校准数据库以用于其他SMET课程；4)对学生和教师使用这一系统的情况进行定量和定性分析，包括仪器对用户的学习和教学策略的影响以及仪器对学生表现的影响。学生在每次作业后使用该工具对他们的学习进行自我评估，明确对他们表现的期望，特别是在概念理解和解决问题方面，并确定提高他们成绩的策略。教师们正在使用它来准备形成性和终结性评估，针对特定的实践和主题来改进教学，确定需要立即帮助的学生群体，并评估新的教学策略和材料的有效性。研究、评价和交流司正在为该项目提供重要的共同资金。

A web-based assessment instrument that provides real-time, multi-dimensional, formative assessment of student learning is being developed. This instrument provides measures of learning by topic and by level of mastery. The levels of mastery are defined as the information level, the algorithmic level, the conceptual level, and the problem solving level. The information level is characterized by memorization and the ability to recall, repeat pieces of information, and identify information that is relevant. The algorithmic level is characterized by the ability to mimic, implement instructions; and use memorized information in familiar contexts. The conceptual level is characterized by the ability to visualize, rephrase, change representations, make connections, and provide explanations. The problem-solving level is characterized by the ability to use material in new contexts; to analyze problems; to identify the information, algorithms, and understanding needed to solve them; to synthesize these components into a solution; and to evaluate the quality of this solution. Four products are resulting from the project: 1) A web-based assessment system founded on a coded database of questions; 2) A coded database of questions for use in measuring student learning in introductory college chemistry, calibrated to reproduce measurements generated through student interviews; 3) Protocols for developing and calibrating databases for use in other SMET courses; and, 4) Quantitative and qualitative analyses of the use of this system by both students and teachers, including the impact of the instrument on the users learning and teaching strategies and the impact of the instrument on student performance. Students are using the instrument to self-assess their learning after each assignment, to clarify expectations for their performance particularly with regard to conceptual understanding and problem solving, and to identify strategies for improving their achievement. Faculty are using it to prepare formative and summative assessments, to target particular practices and topics for improved instruction, to identify groups of students who need immediate assistance, and to assess the effectiveness of new teaching strategies and materials. Important co-funding of this project is being provided by the Division of Research, Evaluation and Communication.