Spoken lAnguage in motions: Learning and Adaptation of speech coMMunication in the context of BOdy motions

在日常生活中，我们经常边说话边移动（例如一起走路，做饭，编织），我们边说话边移动（例如头，手臂和手，姿势）。呼吸在言语和运动之间起着中介作用，是这些活动的来源：它为大脑和肌肉提供所需的氧气，而呼气气流则是产生声音所必需的。呼吸也是语言流动的起搏器：吸气停顿与韵律和句法相协调。根据最近的理论，在现场和具体的认知和语言，Salambo的独创性将考虑肢体运动作为一个共同的背景下，口语，和呼吸之间的调解人肢体和口语。它将采用跨学科的方法，融合语言学，运动科学和心理学。第一个目标是创建一个新的多模态语料库，同时记录肢体运动，呼吸，发音和声学数据使用先进的技术。将采用跨语言纵向方法。 以法语和德语为母语的人将在三个不同的日子里阅读和复述故事，包括小说。已知影响呼吸和肢体运动的说话者的特质属性，即身体健康和身体形状，将被考虑作为身体运动背景下的语音产生的决定性因素。为了进一步评估言语-呼吸-肢体联系，将在不同的运动条件下执行言语任务，其中没有运动、自由手运动以及腿或手的有节奏运动。基于这一语料库，我们将从四个方面来分析口语、呼吸和肢体运动之间的联系：（1）肢体运动、呼吸以及从语音到句法的不同语言层次的特殊物理属性的影响;（B）肢体运动对言语规划、韵律和音段属性的影响; c）使用同步的复杂时间序列分析的言语、呼吸和肢体运动之间的协调;以及d）肢体运动对于新信息和词汇的短期和长期学习的作用。参与这两个团队的研究人员拥有多学科的背景，并在语言和认知科学、语音生成、多模式通信、运动控制和学习、工程和信号处理方面的专业知识相互补充。提出了不同的步骤，将研究结果传播给不同的科学和临床社区以及更广泛的公众。该项目的结果确实将对基础研究产生影响，更深入地了解身体运动背景下的口语，但也将有助于语言和言语教育和治疗的应用研究。

In everyday situations, we often speak while moving (e.g. walking together, cooking, knitting), and we move while speaking (e.g. head, arms and hands, posture). Breathing mediates between speech and motion, and is a source for these activities: it provides the brain and muscles with the required oxygen, and the expiratory airflow is needed for the production of sound. Breathing is also a pacemaker for the speech flow: inhalation pauses are coordinated with prosody and syntax. Following recent theories on situated and embodied cognition and language, Salammbo's originality will be to consider limb motion as a common context for spoken language, and breathing as a mediator between limbs and spoken language. It will adopt an interdisciplinary approach integrating linguistics, movement science and psychology. The first aim is to create a novel multimodal corpus, with simultaneous recordings of limb motion, respiratory, articulatory and acoustic data using advanced technology. A cross-linguistic longitudinal approach will be adopted. Native speakers of French and German will read and retell stories including novel words on three different days. Idiosyncratic properties of the speakers known to influence breathing and limb movements, i.e. physical fitness and body shape, will be taken into account as determinant factors of speech production in the context of body motion. To further assess the speech-breathing-limb link, speech tasks will be performed in different movement conditions with no motion, free hand motion, and rhythmic motions of the legs or hands. Based on this corpus, the link between spoken language, breathing and limb motion will be addressed in four working packages analyzing: a) the impact of idiosyncratic physical properties on limb motion, respiration and different linguistic levels ranging from phonetics to syntax; b) the impact of limb motions on speech planning, prosodic and segmental properties; c) the coordination between speech, breathing and limb motion using sophisticated time series analyses of synchronizations; and d) the role of limb movements for short- and long-term learning of novel information and vocabulary. The researchers involved in the two teams have multi-disciplinary profiles and complement each other with expertise in language and cognitive sciences, speech production, multi-modal communication, motor control and learning, engineering and signal processing. Different steps are proposed to spread the findings to different scientific and clinical communities and to a broader public. The results of the project will indeed have an impact on fundamental research with a deeper understanding of spoken language in the context of body motions, but will also be useful for applied research in language and speech education and therapy.