Measurement and analysis of paralinguistic information and motion for psychotherapy with rich sensibility

在心理治疗师的培训计划中，讲师主要根据他们的经验传授各种技巧，这些技巧可以用书面语言注释。然而，要解释像点头、韵律和态度这样的非语言因素并不容易，这些因素可能携带着对同情和相互信任重要的情感信息。在本研究中，我们试图通过对语音和动作的测量来量化情绪因素，作为训练中有用的信息。我们研究了三个问题：(1)记录心理治疗过程中的视频、语音和头部运动，并将其与数据库中的注释一起存档；(2)数据记录和情绪状态评估工具的开发；(3)分析语音和点头数据以提取规律性。我们收集了44个对话，这些对话对于未来的研究来说是惊人的大。我们开发了一个简单、便携的记录系统和用于情绪状态评估的EMO(情绪运动观察)系统。

In training programs for psychotherapists, the instructors teach various techniques mainly based on their experience which could be annotated with written language. It is, however, not easy to explain nonlinguistic factors like nodding, prosody, and attitude which may carry emotional information important for sympathy and mutual trust. In this study, we tried to quantify the emotional factors based on the measurement of speech and motion as usable information in the training. We studied three issues: (1) Recording video, speech, and head motion in psychotherapy sessions and archiving them with annotations in the database, (2) Development of tools for data recording and emotional state evaluation, (3) Analysis of speech and nodding data to extract regularity. We collected forty four dialogues which are marvelously large for future studies. We developed a simple, portable recording system and EMO (EMOtional Movement Observation) system for emotional state evaluation.