Automated Collaboration Assessment Using Behavioral Analytics

点击翻译按钮获取中文摘要

The Automated Collaboration Assessment Using Behavioral Analytics project willmeasure and support collaboration as students engage in STEM learning activities.Collaboration promotes clarifications of misconceptions and deeper understanding of conceptsin STEM which prepares students for future employment in STEM and beyond. This projectaligns with the goal of the Cyberlearning for Work at the Human-Technology Frontier program tofund exploratory research that supports learners in working productively in technology-richSTEM environments. Collaboration is an important learning skill in K-12 STEM education, yet teachers have fewconsistent ways to measure and support students’ development in this area. This project willresult in both an improved understanding of productive collaboration and a prototypeinstructional tool that can help teachers identify nonverbal behaviors and assess overallcollaboration and engagement quality. Using nonverbal behaviors to assess engagement willdecrease dependence on discourse and content-based dialogue and increase the transferabilityof this work into different domains. This project is particularly timely as the ability to collaborateand engage in group work are growing requirements in professional and learning settings; at thesame time the very act of collaboration is being disrupted by the Coronavirus pandemic andthere is a high likelihood that much of this “new normal” (social distancing; combining in-personand remote collaboration) will be with us for some time. This project will meet the urgent needcurrently felt by educators and educational institutions to support the development ofcollaboration skills among students, even as the very act of collaboration is shifting and nontraditionalforms of education are taking hold.This project is a collaboration between the Center for Education Research andInnovation (CERI) and Center for Vision Technology (CVT) at SRI International (SRI) and willcapture multiple students’ actions as they work collaboratively face-to-face, both in-person andthrough a virtual platform. This project will use a collaboration conceptualmodel, multistage predictive and explainable machine learning models, and video analytics toassess and report on collaborative behaviors and interactions. The behavior analytics systemwill use facial expressions, body movements, and meta-information about the collaboration taskto identify interactions that show how students contribute to the collaboration, individually andcollectively. This 2-year project will use reliability and model prediction testing and sequential,correlation, and thematic analyses of video recordings, surveys, interviews, and student artifactsto answer the following research questions: Can machine learning models reliably assesscollaboration when compared to human assessments? How do individual behaviors duringcollaboration lead and relate to group level interactions and collaboration quality? and Can wevalidate and relate the assessed collaboration behaviors to student outcomes as represented bygroup-generated artifacts? The intellectual merits include contributions to the advancement oftwo fields: (1) machine learning— by developing and exploring new algorithms that generateexplainable collaboration skill assessments and teacher/student dashboards at different grainsizes of the interactions, and (2) learning sciences—by contributing a collaboration conceptualmodel that shows how specific skills, interactions, and behaviors correspond to collaborationquality at group and individual levels. Broader impacts of this work include increasing theavailability and types of feedback presented to instructors and learners from diversebackgrounds. This will expand the settings and number of individuals who can be evaluated andsupported on collaboration by making collaborative learning easier to monitor through tools thatcan be used by a wide audience of educators and professionals.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.