Spoken lAnguage in motions: Learning and Adaptation of speech coMMunication in the context of BOdy motions

动作中的口语：身体动作背景下言语交流的学习和适应

• 批准号: 390200946
• 负责人: Dr. Susanne Fuchs
• 金额: --
• 依托单位: Leibniz-Zentrum Allgemeine Sprachwissenschaft (ZAS)
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2018
• 资助国家: 德国
• 起止时间: 2017-12-31 至 2022-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/390200946?language=en
• 关键词: Spoken; lAnguage; motions; Learning; Adaptation

In everyday situations, we often speak while moving (e.g. walking together, cooking, knitting), and we move while speaking (e.g. head, arms and hands, posture). Breathing mediates between speech and motion, and is a source for these activities: it provides the brain and muscles with the required oxygen, and the expiratory airflow is needed for the production of sound. Breathing is also a pacemaker for the speech flow: inhalation pauses are coordinated with prosody and syntax. Following recent theories on situated and embodied cognition and language, Salammbo's originality will be to consider limb motion as a common context for spoken language, and breathing as a mediator between limbs and spoken language. It will adopt an interdisciplinary approach integrating linguistics, movement science and psychology. The first aim is to create a novel multimodal corpus, with simultaneous recordings of limb motion, respiratory, articulatory and acoustic data using advanced technology. A cross-linguistic longitudinal approach will be adopted. Native speakers of French and German will read and retell stories including novel words on three different days. Idiosyncratic properties of the speakers known to influence breathing and limb movements, i.e. physical fitness and body shape, will be taken into account as determinant factors of speech production in the context of body motion. To further assess the speech-breathing-limb link, speech tasks will be performed in different movement conditions with no motion, free hand motion, and rhythmic motions of the legs or hands. Based on this corpus, the link between spoken language, breathing and limb motion will be addressed in four working packages analyzing: a) the impact of idiosyncratic physical properties on limb motion, respiration and different linguistic levels ranging from phonetics to syntax; b) the impact of limb motions on speech planning, prosodic and segmental properties; c) the coordination between speech, breathing and limb motion using sophisticated time series analyses of synchronizations; and d) the role of limb movements for short- and long-term learning of novel information and vocabulary. The researchers involved in the two teams have multi-disciplinary profiles and complement each other with expertise in language and cognitive sciences, speech production, multi-modal communication, motor control and learning, engineering and signal processing. Different steps are proposed to spread the findings to different scientific and clinical communities and to a broader public. The results of the project will indeed have an impact on fundamental research with a deeper understanding of spoken language in the context of body motions, but will also be useful for applied research in language and speech education and therapy.

在日常生活中，我们经常边走边说(如一起散步、做饭、编织)，边说边动(如头、胳膊、手、姿势)。呼吸是言语和动作之间的中介，是这些活动的来源：它为大脑和肌肉提供所需的氧气，呼气气流是产生声音所必需的。呼吸也是语流的起搏器：吸气停顿与韵律和句法相协调。根据最新的情境和体身认知和语言理论，Salammbo的独创性将是将肢体运动视为口语的共同语境，并将呼吸视为肢体和口语之间的中介。它将采用一种融合语言学、运动学和心理学的跨学科方法。第一个目标是创建一个新的多模式语料库，使用先进的技术同时记录肢体运动、呼吸、关节和声学数据。将采取跨语言的纵向方法。以法语和德语为母语的人将在三个不同的日子里阅读和复述包括新词在内的故事。已知的影响呼吸和肢体运动的说话者的特殊性质，即身体健康和身体形状，将作为身体运动背景下言语产生的决定因素被考虑在内。为了进一步评估语音-呼吸-肢体连接，语音任务将在不同的运动条件下执行，包括无运动、自由手运动和腿部或手的有节奏的运动。在这个语料库的基础上，口语、呼吸和肢体运动之间的联系将被分成四个工作包来分析：a)特殊的物理特性对肢体运动、呼吸和从语音到句法的不同语言水平的影响；b)肢体运动对语音规划、韵律和节段特性的影响；c)使用复杂的同步时间序列分析语音、呼吸和肢体运动之间的协调；以及d)肢体运动在短期和长期学习新信息和词汇方面的作用。参与这两个团队的研究人员拥有多学科背景，并在语言和认知科学、言语产生、多模式交流、运动控制和学习、工程和信号处理方面的专业知识方面相互补充。建议采取不同的步骤将这些发现传播到不同的科学和临床社区以及更广泛的公众。该项目的成果确实将对基础研究产生影响，加深对身体运动背景下的口语的理解，但也将有助于语言和言语教育和治疗的应用研究。