Multimodal Modeling of Coordination and Coregulation Patterns in Speech Rate during Triadic Collaborative Problem Solving
Multimodal Modeling of Coordination and Coregulation Patterns in Speech Rate during Triadic Collaborative Problem Solving
复制标题
DOI：
10.1145/3242969.3242989
复制
发表时间：
2018-10
期刊：
影响因子：
--
通讯作者：
Angela E. B. Stewart;Z. Keirn;S. D’Mello
中科院分区：
文献类型：
--
作者：
Angela E. B. Stewart;Z. Keirn;S. D’Mello
我们在33个黑社会参与协作解决一个具有挑战性的计算机编程任务约20分钟的协调和协同调节模式的模型。我们的目标是前瞻性地对一个队友（A或B或C）的语速（字/秒）（轮流和积极参与的重要信号）从其他两个队友（即，A + B → C; A + C → B; B + C → A）和任务相关的语境特征。我们使用组级嵌套交叉验证训练前馈神经网络（FFNN）和长短期记忆递归神经网络（LSTM）。LSTM的性能优于FFNN和随机基线，可以预测未来6秒的语音速率。一个多模态的语音速率，声学韵律和任务上下文功能的组合优于单峰和双峰信号。在何种程度上，该模型可以预测一个人的语速是正相关的个人的分数在随后的后测，协调/协同调节和协作学习成果之间的联系。我们讨论的实时系统，监控协作过程和干预，以促进积极的协作成果的模型的应用。
We model coordination and coregulation patterns in 33 triads engaged in collaboratively solving a challenging computer programming task for approximately 20 minutes. Our goal is to prospectively model speech rate (words/sec) - an important signal of turn taking and active participation - of one teammate (A or B or C) from time lagged nonverbal signals (speech rate and acoustic-prosodic features) of the other two (i.e., A + B → C; A + C → B; B + C → A) and task-related context features. We trained feed-forward neural networks (FFNNs) and long short-term memory recurrent neural networks (LSTMs) using group-level nested cross-validation. LSTMs outperformed FFNNs and a chance baseline and could predict speech rate up to 6s into the future. A multimodal combination of speech rate, acoustic-prosodic, and task context features outperformed unimodal and bimodal signals. The extent to which the models could predict an individual's speech rate was positively related to that individual's scores on a subsequent posttest, suggesting a link between coordination/coregulation and collaborative learning outcomes. We discuss applications of the models for real-time systems that monitor the collaborative process and intervene to promote positive collaborative outcomes.