Individualized feedback in computer-assisted spoken language learning

该项目的总体目标是通过将内容、反馈和练习改编成外语言语层面的个别学习者，使计算机辅助语言学习软件的可用性和效率达到数量级。这将通过整合学习者自己的语言的产生和感知并从两个方向关注法语-德语对来实现。学习者所经历的困难取决于他们母语的音系，因此是不对称的。该项目的一个特别优势是让在两种语言的语音和音系学方面有广泛工作的语音科学家参与进来，特别是在跨语言环境中。为学习者提供反馈是外语学习的核心问题。第一个目标是为学习者提供自动反馈，这些反馈来自对学习者话语的分析，特别是针对需要改进的声学特征。第二个目标是提供依赖于包含在学习系统中的语音知识并与自动语音识别(ASR)和信号处理相关地交互的反馈。这意味着将考虑对ASR返回的结果的置信度来得出诊断和反馈：置信度越高，反馈的水平越明确。第三个目标是达到用户界面、练习类型、反馈的性质和水平(文本、听觉、视觉)的高度个性化，并制定学习者概况，同时考虑到个别学习者在产生和感知水平上所经历的语音困难。一个关键的方面是使用学习者自己的声音。在培训期间，学习者将接触到他/她自己的作品的修改版本。这将使学习者能够通过将他/她自己的认识与系统提出的反馈进行比较来衡量学习进展。这一目标要求两种语言的语言科学家对L1-L2语言对预测的语音困难有一个准确的定义。这些困难涉及音段水平(例如，停止吸气、最终发音、元音数量)和超音段水平(例如，词汇重音、对比重音)。目标特征将通过考虑两种语言的语音和语音系统来选择，并根据学习者观察到的反复出现的错误进行排序，以确保相关的学习进度。这一过程将通过对语料库的仔细分析来推动，语料库包括由不同熟练程度的学习者记录的法语和德语第二语言语音数据。该语料库将提供给科学界。该项目将依赖于语音学/语音科学和信号处理/语音识别之间的紧密互动，这对这里提出的项目来说是必要的，也是预计将是有效的。

The overall aim of the project is to gain an order of magnitude in the usability and efficiency of computer-assisted language learning software by adapting content, feedback and exercises to individual learners in the speech dimension of a foreign language. This will be achieved by integrating the production and perception of the learner's own speech and focusing on the French-German language pair, in both directions.Difficulties experienced by learners depend on the phonology of their native language and are thus non-symmetric. A particular strength of the project is to involve speech scientists who have worked extensively on the phonetics and phonology of the two languages, and especially in cross-linguistic contexts.Providing learners with feedback is a central issue of foreign language learning. The first objective is to provide the learner with automated feedback which derives from an analysis of the learner's utterance and targets specifically the acoustic features to be improved. The second objective is to offer feedback that relies on phonetic knowledge incorporated in the learning system and interacts relevantly with automatic speech recognition (ASR) and signal processing. This means that confidence in the results returned by ASR will be taken into account to derive diagnoses and feedback: the higher the confidence the more explicit the level of feedback.The third objective is to reach a high level of individualization of the user interface, type of exercises, nature and level of feedback (textual, acoustic, visual) and to develop a learner profile, taking into account the phonetic difficulties experienced by the individual learner on the levels of production and perception. A key aspect consists of using the learner's own voice.During a training session, the learner will be exposed to modified versions of his/her own production. This will enable the learner to measure the learning progress by a comparison of his/her own realizations with the feedback proposed by the system. This objective requires a precise definition of the phonetic difficulties predicted for the L1-L2 pair by speech scientists of both languages. These difficulties concern the segmental level (e.g., stop aspiration, final devoicing, vowel quantity) and the suprasegmental level (e.g., lexical stress, contrastive accent). The target features will be selected by considering the phonological and phonetic systems of both languages and ranked by observed recurring errors made by learners, to ensure a relevant learning progress. This procedure will be driven by a careful analysis of a corpus comprising French and German L2 speech data recorded by learners at different levels of proficiency. This corpus will be made available to the scientific community.The project will rely on a tight interaction between phonetics/speech science and signal processing/speech recognition, which is both necessary and expected to be efficient for the project proposed here.