Spoken lAnguage in motions: Learning and Adaptation of speech coMMunication in the context of BOdy motions

在日常生活中，我们经常边说边动（例如一起走路、做饭、编织），我们边说边动（例如头、胳膊和手、姿势）。呼吸介于语言和运动之间，是这些活动的来源：它为大脑和肌肉提供所需的氧气，呼气气流是产生声音所必需的。呼吸也是语言流的起搏器：呼吸的停顿与韵律和语法相协调。根据最近关于情境和具身认知和语言的理论，salambo的独创性将把肢体运动视为口语的共同语境，而呼吸是肢体和口语之间的中介。它将采用跨学科的方法，整合语言学，运动科学和心理学。第一个目标是创建一个新的多模态语料库，使用先进的技术同时记录肢体运动、呼吸、发音和声学数据。我们将采用跨语言纵向研究方法。以法语和德语为母语的人将在三个不同的日子里阅读和复述故事，包括新单词。说话者已知的影响呼吸和肢体运动的特质，即身体健康和身体形状，将被考虑为身体运动背景下言语产生的决定因素。为了进一步评估言语-呼吸-肢体的联系，言语任务将在不同的运动条件下进行，包括不运动、自由的手运动和腿或手的有节奏的运动。基于该语料库，我们将在四个工作包中分析口语、呼吸和肢体运动之间的联系：a)特殊物理特性对肢体运动、呼吸和不同语言水平（从语音到句法）的影响；B)肢体运动对言语规划、韵律和分词特性的影响；C)语言、呼吸和肢体运动之间的协调，使用复杂的同步时间序列分析；d)肢体运动对短期和长期学习新信息和词汇的作用。这两个团队的研究人员拥有多学科背景，在语言和认知科学、语音生成、多模态通信、运动控制和学习、工程和信号处理等方面的专业知识相互补充。提出了不同的步骤，以将研究结果传播给不同的科学和临床社区以及更广泛的公众。该项目的研究结果将对基础研究产生影响，使人们更深入地了解身体运动背景下的口语，但也将对语言和言语教育和治疗的应用研究有用。

In everyday situations, we often speak while moving (e.g. walking together, cooking, knitting), and we move while speaking (e.g. head, arms and hands, posture). Breathing mediates between speech and motion, and is a source for these activities: it provides the brain and muscles with the required oxygen, and the expiratory airflow is needed for the production of sound. Breathing is also a pacemaker for the speech flow: inhalation pauses are coordinated with prosody and syntax. Following recent theories on situated and embodied cognition and language, Salammbo's originality will be to consider limb motion as a common context for spoken language, and breathing as a mediator between limbs and spoken language. It will adopt an interdisciplinary approach integrating linguistics, movement science and psychology. The first aim is to create a novel multimodal corpus, with simultaneous recordings of limb motion, respiratory, articulatory and acoustic data using advanced technology. A cross-linguistic longitudinal approach will be adopted. Native speakers of French and German will read and retell stories including novel words on three different days. Idiosyncratic properties of the speakers known to influence breathing and limb movements, i.e. physical fitness and body shape, will be taken into account as determinant factors of speech production in the context of body motion. To further assess the speech-breathing-limb link, speech tasks will be performed in different movement conditions with no motion, free hand motion, and rhythmic motions of the legs or hands. Based on this corpus, the link between spoken language, breathing and limb motion will be addressed in four working packages analyzing: a) the impact of idiosyncratic physical properties on limb motion, respiration and different linguistic levels ranging from phonetics to syntax; b) the impact of limb motions on speech planning, prosodic and segmental properties; c) the coordination between speech, breathing and limb motion using sophisticated time series analyses of synchronizations; and d) the role of limb movements for short- and long-term learning of novel information and vocabulary. The researchers involved in the two teams have multi-disciplinary profiles and complement each other with expertise in language and cognitive sciences, speech production, multi-modal communication, motor control and learning, engineering and signal processing. Different steps are proposed to spread the findings to different scientific and clinical communities and to a broader public. The results of the project will indeed have an impact on fundamental research with a deeper understanding of spoken language in the context of body motions, but will also be useful for applied research in language and speech education and therapy.