SMART - Shaping Multilingual Access through Respeaking Technology

SMART - 通过复述技术塑造多语言访问

• 批准号: ES/T002530/1
• 负责人: Elena Davitti
• 金额: $ 59.99万
• 依托单位: University of Surrey
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2020
• 资助国家: 英国
• 起止时间: 2020 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=ES%2FT002530%2F1
• 关键词: SMART; Shaping; Multilingual; Access; through

Our day and age is characterised by a proliferation of live multimedia and multilingual content, such as news and TV programmes. Such content is not accessible to everyone. In many countries, live subtitles in the same language are required by law to enable deaf and hard-of-hearing audiences to enjoy access to information, culture and entertainment. In the UK and other countries (e.g. Canada, Spain, Switzerland), intralingual respeaking is the most well-established technique to produce these subtitles: it relies on human-machine interaction whereby a respeaker listens to the original sound of a live programme or event and simultaneously dictates it to a speech recognition software that turns the each sentence into subtitles displayed on screen.Respeaking so far has been used to produce intralingual subtitles, i.e. in the same language. Given the current multilingual content boom, the SMART project aims to investigate whether respeaking can be used to produce interlingual subtitles, i.e. in a different language. This entails adding translation to an already challenging task which includes listening, speaking, adding oral punctuation, controlling prosody to minimise speech recognition errors, and accounting for space constraints to produce meaningful subtitles. Automatic translation alone is unable to provide live subtitles in a different language of sufficient quality. Interlingual respeaking (IRSP) represents therefore an innovative technique that builds on its monolingual predecessor to achieve this goal. This technique, however promising, has not been systematically tested yet. Therefore there is a need for empirical data to assess its viability. To this end, SMART sets out to achieve three main goals:1. Investigate the feasibility of IRSP by analysing its challenges. Participants' performance during IRSP tasks will be studied in detail to inform optimal strategies to overcome these challenges. Two pilot projects on students have yielded promising results in this respect. We will need to explore these further in a sample of professionals;2. Measure the quality of the live subtitles thus produced using an existing, recently-developed measurement tool (NTR model) which could represent a first step towards establishing a quality benchmark for IRSP;3. Identify the key competencies needed by professionals already working in the language industry (e.g. interpreting, subtitling) to support timely and efficient acquisition of IRSP skills.IRSP has the potential to provide interlingual subtitles of virtually any live events, such as lectures, conferences, theatre shows, business meetings, live TV interviews or programmes involving speakers of different languages. IRSP could also enhance traditional subtitling methods for pre-recorded programmes, improving productivity. By doing so, IRSP would make such events and programmes accessible to everyone, irrespective of any sensory or language barriers.SMART research findings will inform the design of bespoke training courses to equip language professionals with optimal skills to ensure high-quality live interlingual subtitling. We will test these materials in a Summer School during the project; after its end, we will continue to offer training that will provide language professionals with a new career path and the possibility to diversify the services they offer. A public engagement event and informative video will raise awareness of IRSP among the wider public and any other stakeholders interested in providing or benefiting from this service.The UK pioneered the development of intralingual respeaking (today used to subtitle 85% of live TV programmes). SMART will provide the data needed for the UK to become a world leader in IRSP. IRSP is a radical evolution in respeaking methodology and a smart move towards meeting current communication needs, whilst contributing to 'barrier-free' access to different domains of public and private life for all members of society.

我们这个时代的特点是实时多媒体和多语言内容的激增，如新闻和电视节目。这样的内容并不是每个人都能访问的。在许多国家，法律要求使用相同语言的现场字幕，以使聋人和听力障碍的观众能够享受信息、文化和娱乐。在英国和其他国家(如加拿大、西班牙、瑞士)，语内重峰是制作这些字幕最成熟的技术：它依靠人机交互，重播者聆听现场节目或事件的原始声音，同时将其口述给语音识别软件，语音识别软件将每句话转换为屏幕上显示的字幕。迄今为止，重峰已被用于制作语内字幕，即相同语言的字幕。鉴于目前的多语种内容热潮，SMART项目旨在调查是否可以使用重新峰化来制作语际字幕，即用不同的语言制作字幕。这需要在一项已经具有挑战性的任务上增加翻译，这项任务包括听、说、添加口头标点、控制韵律以将语音识别错误降至最低，以及考虑到空间限制以生成有意义的字幕。仅靠自动翻译无法提供具有足够质量的不同语言的实时字幕。因此，语际重峰(IRSP)代表了一种创新的技术，它建立在单语前身的基础上，以实现这一目标。这项技术尽管前景看好，但还没有经过系统的测试。因此，需要经验数据来评估其可行性。为此，SMART着手实现三个主要目标：1.通过分析IRSP面临的挑战，研究其可行性。将详细研究参与者在IRSP任务中的表现，以确定克服这些挑战的最佳战略。关于学生的两个试点项目在这方面取得了可喜的成果。我们将需要在专业人员样本中进一步探讨这些问题；2.使用现有的、最近开发的测量工具(NTR模型)来衡量由此产生的现场字幕的质量，这可能是为IRSP建立质量基准的第一步；3.确定已经在语言行业工作的专业人员所需的关键能力(例如口译、字幕)，以支持及时和有效地获得IRSP技能。IRSP有可能为几乎任何现场活动提供语际字幕，例如讲座、会议、戏剧表演、商务会议、直播电视采访或涉及不同语言的节目。IRSP还可以改进传统的预先录制节目的字幕方法，提高生产率。通过这样做，IRSP将使每个人都可以访问这些活动和节目，而不受任何感官或语言障碍的影响。SMART的研究结果将为定制培训课程的设计提供参考，以使语言专业人员掌握最佳技能，以确保高质量的跨语种字幕现场直播。在项目期间，我们将在暑期学校测试这些材料；项目结束后，我们将继续提供培训，为语言专业人员提供一条新的职业道路，并使他们提供的服务多样化。公众参与活动和信息丰富的视频将提高更广泛的公众和任何其他有兴趣提供这项服务或受益于这项服务的利益相关者对IRSP的认识。英国率先发展了语内重峰(目前用于为85%的电视直播节目加字幕)。SMART将提供英国成为IRSP世界领先者所需的数据。IRSP是重新确定方法的一次彻底演变，是满足当前通信需求的明智举措，同时有助于社会所有成员无障碍地进入公共和私人生活的不同领域。

项目成果

Hybrid workflows for real-time interlingual communication via speech recognition: the SMART project

通过语音识别进行实时跨语言交流的混合工作流程：SMART 项目

• 发表时间: 2022
• 作者: Davitti Elena

Real-time interlingual speech-to-text via speech recognition: mapping the field of a surging accessibility service

通过语音识别实现实时跨语言语音转文本：绘制蓬勃发展的无障碍服务领域

• 发表时间: 2020
• 作者: Davitti E

Hybrid workflows for live interlingual speech-to-text communication: placing the human at the core

实时语言间语音到文本通信的混合工作流程：以人为本

• 发表时间: 2022
• 作者: Davitti Elena

Respeaking e il progetto SMART - un contributo sperimentale allo studio del respeaking interlinguistico. (Respeaking and the SMART project: an experimental contribution to the study of interlingual respeaking)

Respeaking e il progetto SMART - un contributo sperimentale allo studio del respeaking interlinguistico.

• 发表时间: 2022
• 作者: Bulgarelli. K.

Shaping Multilingual Access Through Respeaking Technology - Project poster

通过复述技术塑造多语言访问 - 项目海报

• 发表时间: 2023
• 作者: Davitti Elena