Unified Transcription And Translation For Extended Reality

扩展现实的统一转录和翻译

• 批准号: 10039436
• 金额: $ 111.01万
• 依托单位: UNIVERSITY OF EDINBURGH
• 依托单位国家: 英国
• 项目类别: EU-Funded
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=10039436
• 关键词: Unified; Transcription; Translation; Extended; Reality

The aim of UTTER is to leverage large language models to build the next generation of multimodal eXtended reality (XR) technologies for transcription, translation, summarisation, and minuting. We will make these technologies scalable, adaptable, contextualised, robust, explainable, and emotion-aware. We will increase the context-sensitivity of the technologies, so they can take into account the full history of the conversation, as well as its wider context. We will introduce confidence-aware models, which can take into account their own limitations. We will develop explainable models, so the human user can know why the model made the decisions it did. We will improve adaptation, so that domain-specific and language-specific models can be quickly rolled out. For these advances we will make use of pretrained eXtended reality (XR) models, which optimally combine text and speech signals, and are trained efficiently with adapters and prompting. We will also develop efficient methods to deploy such large and complex models, so that they can put into production in an energy-efficient manner. Our use-case prototypes will cover (i) A personal assistant for meetings that can improve communication in the online world and (ii) an advanced customer service assistant to support global markets. These prototypes will be developed and tested throughout the project, with annual releases and evaluations. Through our cascaded grant programme, and our release of tools to facilitate the use of pre-trained XR models, will enable the take-up and development of these technologies throughout Europe.

OUTER的目标是利用大型语言模型构建下一代多模式扩展现实(XR)技术，用于转录、翻译、摘要和记录。我们将使这些技术具有可伸缩性、适应性、情境化、健壮性、可解释性和情感感知。我们将提高技术的上下文敏感度，以便它们可以考虑对话的完整历史以及其更广泛的背景。我们将引入信心感知模型，该模型可以考虑到它们自身的局限性。我们将开发可解释的模型，这样人类用户就可以知道模型为什么做出这样的决定。我们将提高适应性，使特定领域和特定语言的模型能够快速推出。对于这些进步，我们将利用预先训练的扩展现实(XR)模型，该模型最佳地结合了文本和语音信号，并通过适配器和提示进行了有效的训练。我们还将开发高效的方法来部署这样的大型复杂模型，以便它们能够高效地投产。我们的用例原型将涵盖(I)可以改善在线世界沟通的会议个人助理，以及(Ii)支持全球市场的高级客户服务助理。这些原型将在整个项目中进行开发和测试，并每年发布和评估。通过我们的级联赠款计划，以及我们发布的促进使用预先培训的XR模型的工具，将使这些技术能够在整个欧洲采用和发展。