RI: EAGER: Collaborative Research: Adaptive Heads-up Displays for Simultaneous Interpretation

口译是将语言从一种语言翻译成另一种语言的任务，是在国际会议、旅行或外交等多语言环境中促进交流的重要工具。然而，同声传译是一项极其困难的任务，需要高水平的经验和训练，因为同声传译必须在说话者说话的同时产生结果。特别是，同声传译人员经常发现某些内容，如技术术语，人员和组织的名称，以及数字特别难以正确翻译。这个探索性研究项目的早期拨款旨在创建自动口译助手，通过识别原始语言的内容，并在平视显示器（类似于提词器）上显示翻译结果，以帮助口译员处理这些难以翻译的内容，以便口译员在需要时使用。这将使同声传译更有效、更容易获得，使跨语言和文化的对话更自然、更普遍、更有效，并使世界各地的社区和文化在贸易、合作和友谊中联系起来。创建这些系统在技术上是一个具有挑战性的问题，以前没有人尝试过。一个挑战是，同声传译已经是一项认知负担的任务，任何界面都不能因为过于侵入而过度增加口译员的认知负担。为了减少这种认知负荷，需要一个能够决定何时提供翻译建议以及何时不提供翻译建议的界面。为了实现这一目标，本项目将开发对语音识别错误具有鲁棒性的方法，并通过观察口译员的口译结果来学习显示什么。所提议的框架的效用将根据它在多大程度上提高口译员产生流利、准确的口译结果的能力，以及额外的界面对他们施加的认知负荷来评估。

Interpretation, the task of translating speech from one language to another, is an important tool in facilitating communication in multi-lingual settings such as international meetings, travel, or diplomacy. However, simultaneous interpretation, during which the results must be produced as the speaker is speaking, is an extremely difficult task requiring a high level of experience and training. In particular, simultaneous interpreters often find certain content such as technical terms, names of people and organizations, and numbers particularly hard to translate correctly. This Early Grant for Exploratory Research project aims to create automatic interpretation assistants that will help interpreters with this difficult-to-translate content by recognizing this content in the original language, and displaying translations on a heads-up display (similar to teleprompter) for interpreters to use if they wish. This will make simultaneous interpretation more effective and accessible, making conversations across languages and cultures more natural, more common, and more effective and joining communities and cultures across the world in trade, cooperation, and friendship.Creating these systems is a technically challenging problem and has not previously been attempted. One challenge is that simultaneous interpretation is already a cognitively taxing task, and any interface must not unduly increase the cognitive load on the interpreter by being too intrusive. Reducing this cognitive load requires an interface that can decide when to provide translation suggestions and when to refrain from doing so. To achieve this goal, this project will develop methods that are robust to speech recognition errors, and learn what to display by observing the interpreters' interpretation results. The utility of the proposed framework will be evaluated with respect to how much it improves the ability of interpreters to produce fluent, accurate interpretation results, as well as the cognitive load the additional interface imposes on them.