Multimodal Stancetaking: Expressive Movement and Affective Stance. Political Debates in the German Bundestag and the Polish Sejm

目的：尽管关于姿势和站姿的文献丰富多样，但迄今为止对姿势和身体动作的作用以及这一交际活动的韵律特征的研究还很少。该项目回应了将多模式纳入制票研究的挑战，并提出了多模式制票的新框架。最初发展起来的一个理论和方法概念是为了解释视听媒体和面对面互动中的隐喻意义产生，表达运动的概念是用来解释和分析作为一种多通道活动如何成为一种时间结构的情感体验的途径。综合方法：在德国联邦议院和波兰议会的政治辩论中对多通道通道进行了研究。重点是动态的手势模式以及身体姿势、头部动作和语音，包括其作为多模式表达动作的韵律特性。在“表现运动单位”的层面上分析了统计。将对单元进行标记，并独立分析手势和韵律形式以及共现语音。这些辩论将被转录、注解，并提交以下方面的分析：(1)表达运动单位的分割，(2)高情感水平的表达形式和模式的分析，(3)根据(A)情感立场，(B)相互情感排列的多模式协调。通过将共语手势的分析与韵律参数与情感或态度的关系研究相结合，为多通道统计分析提供了一个新的视角。预期影响：该项目在以下方面取得了新的突破：(1)提供了一种作为富有表现力的运动的多模式统计的新方法，(2)通过包括手势-语音-韵律同步来补充现有的关于多模式统计的研究，以及(3)调查两种语言(德语和波兰语)政治辩论中的多模式统计。因此，该项目为今后跨文化、跨话语体裁或不同形式的政治辩论的多式联运的比较研究提供了基础，并将开放获取结果和方法。为了捕捉多模式形式的动态复杂性和作为情感形式的停顿模式，即作为多模式“表达运动”及其在政治辩论中的协调，该项目整合了两个独特的专家团队。波兰团队在语音韵律分析方面处于领先地位，而德国团队在语言手势分析和关键概念表达运动的应用方面处于领先地位。通过整合这两种类型的专业知识，该项目提供了一种独特的视角，将Stanceting作为多模式的表达运动。

Objective: Despite the abundant and diverse literature on stance and stancetaking the role of gesture and body movement as well as prosodic properties of this communicative activity remain hitherto only scarcely studied. The project responds to the challenge of including multimodality in stancetaking research and advances a new framework to multimodal stancetaking. Initially developed as a theoretical and methodological concept to account for metaphorical meaning-making in audiovisual media and in face-to-face interaction, the notion of 'expressive movement' is the pathway offered to explain and analyze how stancetaking as a multimodal activity becomes a temporally-structured affective experience.Integrating Methods: Multimodal stancetaking is investigated in political debates in the German Bundestag and the Polish Sejm. The focus is on dynamic gestural patterns along with body-postures, head movements, and speech, including its prosodic properties as multimodal expressive movements. Stancetaking is analyzed on the level of 'expressive movement units'. Units will be tagged and independently analyzed for gestural and prosodic form as well as for co-occurring speech. The debates will be transcribed, annotated, and submitted to analyses with regard to: (1) segmentation of expressive movement units, (2) analysis of expressive forms and patterns with high levels of affectivity, (3) multimodal orchestration of affective stancetaking in terms of (a) affective stance, (b) interaffective alignment. By integrating the analysis of co-speech gesture with prosodic parameters that are relatively well explored in terms of their relation to emotions or attitudes, a new angle is provided on the analysis of multimodal stancetaking. Expected Impact: The project breaks new ground by: (1) offering a new approach to multimodal stancetaking as expressive movement, (2) complementing existing research on multimodal stancetaking by including gesture-speech-prosody synchronization, and by (3) investigating multimodal stancetaking in political debates across two languages (German and Polish). The project thus provides the basis for future comparative research of multimodal stancetaking, be it across cultures, discourse genres, or different forms of political debates, and will offer open access to results and methodology.Added value of the international cooperation. To capture the dynamic complexity of multimodal forms and patterns of stancetaking as affective forms, i.e., as multimodal 'expressive movements' and their orchestration along political debates, the project integrates two unique expert teams. While the Polish team takes the lead in speech prosody analyses, the German team leads with regard to linguistic gesture analysis and application of the key concept of 'expressive movement'. By integrating the two types of expertise the project offers a unique perspective on stancetaking as multimodal expressive movement.