Matching Data-Driven Models of Group Interactions to Video Analysis of Collaborative Problem Solving on Tablet Computers
Matching Data-Driven Models of Group Interactions to Video Analysis of Collaborative Problem Solving on Tablet Computers
复制标题
将群体交互的数据驱动模型与平板电脑上协作解决问题的视频分析相匹配
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Yurui Tong
中科院分区：
文献类型：
--
作者：
L. Paquette;Nigel Bosch;Emma Mercier;Jiyoon Jung;Saadeddine Shehab;Yurui Tong
尽管越来越强调在课堂上使用协作学习，但关于如何成功地实施它，仍有很多需要了解的地方。特别是教师在协作学习活动中应该扮演什么样的角色，如何更好地支持和指导教师开展协作学习活动，目前还不清楚。在这项研究中，我们研究了如何利用数字学习环境，通过匹配视频和日志数据，通过数据驱动的学生协作互动模型来支持协作学习。当学生使用协作素描工具解决问题时，模型成功地检测出了任务外行为（比机会水平准确率高出43.2%）和任务相关的谈话（比机会水平高出34.5%）。未来的工作将研究如何使用这些模型，使教师能够有效地干预，通过使用数据驱动的工具来支持协作学习，这些工具将为他们提供有关学生行为的实时信息。协作解决问题是一项重要的技能（Hesse, Care, Buder, Sassenberg, & Griffin, 2015），其在国际教育和评估系统中的地位日益突出（例如ABET, 2015； NRC, 2012； OECD, 2017）。然而，关于如何在课堂上成功实施协作学习，还有很多需要理解的地方（Nokes-Malach, Richey, & Gadgil, 2015），特别是教师如何最有效地支持学生的互动（例如Webb等人，2009）。Kaendler及其同事（2015）确定了教师在群体斗争时监控和干预方面发挥的关键作用，先前的工作表明教师干预的相关性对成功的群体结果很重要（例如Dekker & Elshout-Mohr， 2004）。然而，虽然大师教师更有可能掌握评估如何以及何时干预的专业知识，但更多的新手教师可能在这方面遇到困难。例如，早期的研究表明，几乎所有研究生助教的干预都以内容为重点，很少有干预关注于支持学生的协作互动（Mercier, Shehab & Kessler，正在审查中）。因此，有必要探索为新手教师提供洞察小组过程的方法，使他们能够更多地了解小组内部发生的事情，并进行适当的干预（例如Alavi和Dillenbourg， 2012）。在本文中，我们介绍了创建一个数据驱动的教师工具的初步工作，该工具可以自动提供这种见解。基于一个为工程专业学生创建共享表示工具的项目，我们通过将学生工具中的日志数据与他们交互行为的视频分析相匹配，开发了预测模型。我们的研究结果表明，使用学生行为日志来评估他们互动的质量是有潜力的，这可以在教师工具中实施，以增加他们的观察，并为何时以及如何最好地干预提供见解。协作学习在学习和迁移方面的价值，以及作为提高STEM领域持久性和兴趣的一种方式，已经在一系列研究中得到了证实（例如Barron 2003； Gasiewski et al., 2012）。然而，在课堂和实验室研究中都记录了结果质量的差异（Nokes-Malach等人，2015），而成功通常与合作期间学生互动的质量有关（Kaendler等人，2015）。Hesse及其同事（2015）确定了与成功互动最相关的行为，并将其分为社交技能和认知技能。越来越多的人认识到，学生需要帮助发展这些技能，仅仅将学生分组是不足以使小组发挥良好作用的（作者，在审查中；Borge & White, 2016）。当学生发展这些技能时，教师在干预支持小组方面发挥着关键作用，他们需要快速评估学生是否需要与社会或认知协作过程有关的支持，或者与课程内容有关的支持。这一领域的初步工作指向通过学生的行为提供富有成效的见解-例如，通过改变灯的颜色（Alavi & Dillenbourg, 2012）或发布tweet (Mercier, Rattray, & Lavery, 2015) -或通过学生使用的软件自动提供给教师的见解（Martinez-Maldonado, Yacef, & Kay, 2015; Mericer, 2016）。因此，我们在本文中的主要问题是如何自动检测学生的协作交互模式，并利用它们为教师提供见解？教育数据挖掘领域的研究人员（Baker & Yacef, 2009）研究了如何使用机器学习方法来构建学生模型，这些模型能够检测使用数字学习环境的学生何时从事特定行为，或推断学生当前的心理状态。这是通过收集学生在学习环境中行为的详细日志来实现的。然后使用机器学习算法分析这些日志，以找到学生行为与建模结构之间的关系。例如，一个模型可能会了解到，在家庭作业问题上反复提交相同的答案是沮丧的表现。动作日志已被用于模拟多种类型的数字学习环境中的各种结构，例如学生在智能辅导系统中的脱离，其中模型被训练以检测学生何时试图“游戏系统”（Baker, Corbett, Roll, & Koedinger, 2008; Paquette, de Carvahlo, Baker, & Ocumpaugh, 2014）。游戏系统是一种非任务行为，学生利用计算机导师的功能猜测答案或让导师提供答案。Sabourin， Mott和Lester（2013）研究了如何使用动作日志来模拟名为《水晶岛》的教育游戏中的自我调节学习行为。Gobert, Sao Pedro， Raziuddin和Baker（2013）使用动作日志来评估学生是否表现出与科学探究技能使用相关的行为。此外，动作日志还被用于检测学生的心理状态，如情感状态（Baker et al., 2012; Kai et al., 2015; Paquette et al., 2014; Pardos, Baker, San Pedro, Gowda, & Gowda, 2014）或是否走神（Mills & D ' mello, 2015）。这种类型的研究已经在许多类型的学习环境中进行，包括智能导师，教育游戏和科学模拟微世界。
Despite an increasing emphasis on the use of collaborative learning in classrooms, there is still much to be understood about how to successfully implement it. In particular, it is still unclear what the role of teachers should be during collaborative learning activities and how we can better support and guide teachers in their implementation of collaborative activities. In this study, we investigated how digital learning environments can be leveraged to support collaborative learning through data-driven models of students’ collaborative interactions by matching video and log data. The models successfully detected off-task behavior (43.2% above chance-level accuracy) and task-related talk (34.5% above chance) as students solved problems using a collaborative sketching tool. Future work will investigate how these models can be used to allow instructors to intervene effectively to support collaborative learning through the use of data-driven tools which will provide them with live information about the students’ behaviors. Major issues and theoretical approaches Collaborative problem solving is an important skill (Hesse, Care, Buder, Sassenberg, & Griffin, 2015), and its prominence in international education and assessment systems has been increasing (e.g. ABET, 2015; NRC, 2012; OECD, 2017). However, there is still much to be understood about how to successfully implement collaborative learning in classrooms (Nokes-Malach, Richey, & Gadgil, 2015), and in particular, how teachers can be most effective in supporting students’ interactions (e.g. Webb et al., 2009). Kaendler and colleagues (2015) identified a key role that teachers play in monitoring and intervening when groups struggle, and prior work indicates the relevance of teacher interventions is important for successful group outcomes (e.g. Dekker & Elshout-Mohr, 2004). However, while master teachers are more likely to have developed expertise about assessing how and when to intervene, more novice teachers may struggle with this. For example, earlier work has shown that almost all interventions made by graduate teaching assistants were content focused, with very few interventions focused on supporting students’ collaborative interactions (Mercier, Shehab & Kessler, under review). Thus, there is a need to explore ways to provide insight into the group processes for novice teachers, allowing them to understand more about what is going on within groups, and intervene appropriately (e.g. Alavi & Dillenbourg, 2012) . In this paper, we present initial work towards creating a data-driven teacher tool that automatically provides such insight. Building on a project that sought to create a shared representation tool for engineering students, we developed prediction models by matching log data from the student tool to video analysis of their interaction behaviors. Our results indicate that there is potential in using logs of student actions to assess the quality of their interactions, which could be implemented in a teacher tool to augment their observations and provide insight into when and how best to intervene. Collaborative learning The value of collaborative learning for both learning and transfer and as a way to increase persistence and interest in STEM fields has been identified across a range of studies (e.g. Barron 2003; Gasiewski et al., 2012). However, variation in the quality of outcome has been recorded in both classroom and laboratory studies (Nokes-Malach et al., 2015), and success is most often associated with the quality of student interactions during collaborations (Kaendler, et al., 2015). Hesse and colleagues (2015) identified behaviors that are most associated with successful interactions, dividing them into social skills and cognitive skills. There is an increasing recognition that students need help developing these skills, and that merely placing students in groups is not sufficient for groups to function well (Authors, under review; Borge & White, 2016). Teachers play a key role in intervening to support groups as they develop these skills, needing to make a quick assessment as to whether students need support in relation to the social or cognitive collaborative processes, or in relation to the course content. Initial work in this area points towards productive insights being provided either by student actions—for example, by changing the color of a lamp (Alavi & Dillenbourg, 2012) or posting a tweet (Mercier, Rattray, & Lavery, 2015)—or by insight automatically provided to the teacher by the software the students are using (Martinez-Maldonado, Yacef, & Kay, 2015; Mericer, 2016). Thus, our primary question in this paper is how can we automatically detect students’ collaborative interaction patterns and use them to provide insight to teachers? Modeling student behavior from action logs Researchers in the field of Educational Data Mining (Baker & Yacef, 2009) have studied how machine learning approaches can be used to build student models that are able to detect when students using digital learning environments are engaging in specific behaviors, or to infer the student’s current state of mind. This is achieved by collecting detailed logs of students’ actions within the learning environment. Those logs are then analyzed using machine learning algorithms to find relationships between the students’ actions and the modeled construct. For example, a model might learn that repeatedly submitting the same answer on a homework problem is indicative of frustration. Action logs have been used to model a variety of constructs across multiple types of digital learning environments, such as students’ disengagement in intelligent tutoring systems, where models were trained to detect when students attempt to “game the system” (Baker, Corbett, Roll, & Koedinger, 2008; Paquette, de Carvahlo, Baker, & Ocumpaugh, 2014). Gaming the system is a type of off-task behavior in which students exploit a computerized tutor’s functionalities to guess an answer or have the tutor provide them with the answer. Sabourin, Mott, and Lester (2013) studied how action logs can be used to model self-regulated learning behaviors in an educational game called Crystal Island. Gobert, Sao Pedro, Raziuddin, and Baker (2013) used action logs to assess whether students were showing behaviors related to the usage of science inquiry skills. In addition, action logs have been used to detect students’ states of mind, such as their affective states (Baker et al., 2012; Kai et al., 2015; Paquette, et al., 2014; Pardos, Baker, San Pedro, Gowda, & Gowda, 2014) or whether they are mind-wandering (Mills & D’Mello, 2015). This type of research has been conducted in many types of learning environments including intelligent tutors, educational games, and science simulation microworlds.