RI: EAGER: Collaborative Research: Adaptive Heads-up Displays for Simultaneous Interpretation

口译是将讲话从一种语言翻译成另一种语言的任务，是促进多语言环境（如国际会议，旅行或外交）中沟通的重要工具。然而，同声传译是一项极其困难的任务，需要大量的经验和培训，因为同声传译必须在发言者发言时产生结果。特别是，同声传译通常会发现某些内容，如技术术语，人员和组织的名称以及数字特别难以正确翻译。这个探索性研究项目的早期资助旨在创建自动翻译助手，通过识别原始语言的内容来帮助口译员处理这些难以翻译的内容，并在平视显示器上显示翻译（类似于电传打字机），以便口译员在需要时使用。这将使同声传译更有效、更容易获得，使跨语言和文化的对话更自然、更常见、更有效，并将世界各地的社区和文化在贸易、合作和友谊中联系起来。创建这些系统是一个技术上具有挑战性的问题，以前没有尝试过。一个挑战是同声传译已经是一项认知繁重的任务，任何界面都不能因过于侵入而不适当地增加口译员的认知负荷。减少这种认知负荷需要一个界面，可以决定何时提供翻译建议，何时不这样做。为了实现这一目标，本项目将开发对语音识别错误具有鲁棒性的方法，并通过观察口译员的口译结果来学习显示什么。所提出的框架的效用将进行评估方面有多少提高口译员的能力，以产生流畅，准确的口译结果，以及认知负荷的额外接口强加给他们。

Interpretation, the task of translating speech from one language to another, is an important tool in facilitating communication in multi-lingual settings such as international meetings, travel, or diplomacy. However, simultaneous interpretation, during which the results must be produced as the speaker is speaking, is an extremely difficult task requiring a high level of experience and training. In particular, simultaneous interpreters often find certain content such as technical terms, names of people and organizations, and numbers particularly hard to translate correctly. This Early Grant for Exploratory Research project aims to create automatic interpretation assistants that will help interpreters with this difficult-to-translate content by recognizing this content in the original language, and displaying translations on a heads-up display (similar to teleprompter) for interpreters to use if they wish. This will make simultaneous interpretation more effective and accessible, making conversations across languages and cultures more natural, more common, and more effective and joining communities and cultures across the world in trade, cooperation, and friendship.Creating these systems is a technically challenging problem and has not previously been attempted. One challenge is that simultaneous interpretation is already a cognitively taxing task, and any interface must not unduly increase the cognitive load on the interpreter by being too intrusive. Reducing this cognitive load requires an interface that can decide when to provide translation suggestions and when to refrain from doing so. To achieve this goal, this project will develop methods that are robust to speech recognition errors, and learn what to display by observing the interpreters' interpretation results. The utility of the proposed framework will be evaluated with respect to how much it improves the ability of interpreters to produce fluent, accurate interpretation results, as well as the cognitive load the additional interface imposes on them.