ERI: IoT-Enabled Smart Learning Environment for Ambient Assessment of Socio-Technical Skills in Engineering Students

该奖项全部或部分由2021年美国救援计划法案（公法117-2）资助，今天的工程师需要同时掌握其领域的技术和社会技术技能，这些技能会显著影响他们的专业表现和他们产生的解决方案。这些技能的例子包括做出道德判断的能力，在团队中有效运作的能力，以及对安全程序的根深蒂固的遵守。虽然教师可以通过考试和项目演示等传统评估方法轻松评估技术知识和技能，但现有的社会技术学习目标评估方法可能具有设计挑战性，管理耗时，主观性不一致，并且向学生反馈缓慢。该项目将设计和开发一种新的基于物联网（IoT）的自动化评估工具，用于评估学生在工程课程中实现社会技术学习目标的情况，这些课程可能会解决这些问题。本项目提出的智能学习环境（SLE）环境评估框架（SAAF）将能够在正常进行的课堂设置（如实验室活动或小组讨论）中检测学生和教师的行为，推理这些行为的背景，并根据课程或活动的学习目标对学生的成绩水平进行定量分类。这可能会改变工程师在社会技术领域的培训方式，因为环境观察可以捕捉更接近学生在工作场所作为专业工程师的行为。这与NSF的工程师形成研究（RFE）计划相一致，因为该项目将开发一种工具，可以有效地帮助开发工程师的专业能力。SAAF的总体目标是对学生在课程的社会技术学习目标方面的表现进行自动评估。SAAF将在常规课堂活动期间应用基于知识的课堂占用者（学生和教师）活动识别，以检测行动并推理这些行动的上下文。将采用基于知识的活动识别方法，以便不像基于机器学习的方法那样，在教学和评估的方式上具有限制性和规定性。 检测到的事件将对课堂活动和课程内容本体通过该项目开发进行分析。然后，该项目将设计和实施SAAF的概念验证，方法是在实验室教室内使用视频、音频和传感器检测事件，并根据上下文进行推理活动，例如“学生A在教师提示之前戴上防静电腕带”，或“在学生B和C的团队中，B完成了所有工作”。随着这一能力的发展，该项目将通过两个研究问题的调查，调查该系统的质量和影响-（1）如何有效，可靠，方便，快捷的SAAF作为一个自动化工具，评估工程专业学生的社会技术技能？以及（2）SAAF作为社会技术学习目标的主要课堂评估工具的主要障碍是什么，如何解决这些障碍以增加购买？为了回答这些问题，学生和教师参与者将被要求进行定期课堂实验室活动在SAAF启用的教室，并比较SAAF评估结果与可比的传统评估方法。还将进行参与者调查和访谈，以分析用户体验。因此，该项目将提供以下主要贡献，每一个都解决了当前工程教育的差距：（1）可共享和可扩展的课堂活动和评估本体，（2）用于自动化环境评估技术的概念验证，以及（3）分析环境评估技术的可靠性，影响，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2)Today’s engineers need to attain both technical mastery of their field and socio-technical skills that significantly impact their professional performance and the solutions they produce. Examples of these skills include the ability to make ethically informed judgments, the ability to function effectively in a team, and an ingrained observance of safety procedures. While technical knowledge and skills can be easily assessed by instructors through traditional assessment methods such as exams and project demonstrations, existing assessment methods for socio-technical learning objectives can be challenging to design, time-consuming to administer, inconsistent in its subjectivity and slow in returning feedback to students. This project will design and develop a novel automated Internet of Things (IoT)-based assessment tool for evaluating student attainment of socio-technical learning objectives in engineering courses that can potentially address these issues. The Smart Learning Environment (SLE) Ambient Assessment Framework (SAAF) proposed in this project will be able to detect student and instructor actions within a normally conducted class setting such as lab activity or group discussion, reason the context of those actions, and produce a quantitative classification of student achievement level with respect to the learning objectives of the course or activity. This could transform the way engineers are trained in socio-technical areas as ambient observation could capture behaviors that more closely resemble how students would behave in the workplace as professional engineers. This is in line with NSF’s Research in the Formation of Engineers (RFE) program as this project will develop a tool that can potentially efficiently and effectively help develop professional competencies of engineers.The SAAF’s overall goal is to perform automated assessment of student performance with respect to a course’s socio-technical learning objectives. SAAF will apply knowledge-based activity recognition of classroom occupants (students and instructors) during a regular class activity to detect actions and reason the context of those actions. A knowledge-based approach to activity recognition will be employed so as not to be restrictive and prescriptive in the way instruction and assessments should be carried out as machine learning-based approaches would. The detected events will be analyzed against classroom activity and course content ontologies developed through the project. The project will then design and implement a proof of concept for the SAAF by employing video, audio and sensor detection of events within a laboratory classroom and reasoning activities with context, such as “student A put on antistatic wrist band before instructor prompted,” or “in student B and C’s team, B did all the work”. With this capability developed, the project will then investigate the quality and impact of this system through the investigation of two research questions – (1) How valid, reliable, convenient, and expedient is the SAAF as an automated tool for assessing socio-technical skills of engineering students? and (2) What are the major barriers to SAAF adoption as a primary classroom assessment tool for socio-technical learning objectives, and how can those barriers be addressed to increase buy-in? To answer these questions, student and instructor participants will be asked to conduct regular class laboratory activities in an SAAF-enabled classroom and compare SAAF assessment results with those of comparable traditional assessment methods. Participant surveys and interviews will also be conducted to analyze user experience. Therefore, the project will provide the following major contributions, each addressing a current gap in engineering education: (1) shareable and extendable classroom activity and assessment ontologies, (2) a proof-of-concept for an automated ambient assessment technology, and (3) data analyzing the reliability, impact, and acceptability of such a system.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.