Dialogue Act Classification via Transfer Learning for Automated Labeling of Interviewee Responses in Virtual Reality Job Interview Training Platforms for Autistic Individuals
Dialogue Act Classification via Transfer Learning for Automated Labeling of Interviewee Responses in Virtual Reality Job Interview Training Platforms for Autistic Individuals
复制标题
DOI：
10.3390/signals4020019
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
Deeksha Adiani;Kelley Colopietro;Joshua W. Wade;Miroslava Migovich;Timothy J. Vogus;N. Sarkar
中科院分区：
文献类型：
--
作者：
Deeksha Adiani;Kelley Colopietro;Joshua W. Wade;Miroslava Migovich;Timothy J. Vogus;N. Sarkar
基于计算机的求职面试培训，包括虚拟现实 (VR) 模拟，近年来越来越受欢迎，以支持和帮助自闭症患者，他们在寻找和维持就业方面面临重大挑战和障碍。尽管很受欢迎，但这些培训系统通常无法模仿就业面试的复杂性和动态性，因为虚拟对话代理的对话管理要么依赖于从预先指定的答案菜单中进行选择，要么对话处理基于从受访者转录的语音中提取关键字，而这取决于面试脚本。我们通过迁移学习的自动对话行为分类来解决这一限制。这允许从用户语音中识别意图，而与采访的领域无关。我们还通过提供原始数据集以及 22 位自闭症参与者在虚拟求职面试平台中对面试问题的回答，解决了一般求职面试对话行为分类器领域缺乏训练数据的问题。参与者对定制采访脚本的回答被转录为文本，并根据定制的 13 类对话方案进行注释。最好的分类器是来自 Transformer (BERT) 模型的微调双向编码器表示，其 f1 分数为 87%。
Computer-based job interview training, including virtual reality (VR) simulations, have gained popularity in recent years to support and aid autistic individuals, who face significant challenges and barriers in finding and maintaining employment. Although popular, these training systems often fail to resemble the complexity and dynamism of the employment interview, as the dialogue management for the virtual conversation agent either relies on choosing from a menu of prespecified answers, or dialogue processing is based on keyword extraction from the transcribed speech of the interviewee, which depends on the interview script. We address this limitation through automated dialogue act classification via transfer learning. This allows for recognizing intent from user speech, independent of the domain of the interview. We also redress the lack of training data for a domain general job interview dialogue act classifier by providing an original dataset with responses to interview questions within a virtual job interview platform from 22 autistic participants. Participants’ responses to a customized interview script were transcribed to text and annotated according to a custom 13-class dialogue act scheme. The best classifier was a fine-tuned bidirectional encoder representations from transformers (BERT) model, with an f1-score of 87%.