Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation
Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation
复制标题
过度生成无法获得奖励：同步语音翻译的长度自适应平均滞后
DOI：
10.18653/v1/2022.autosimtrans-1.2
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Marco Turchi
中科院分区：
文献类型：
--
作者：
Sara Papi;Marco Gaido;Matteo Negri;Marco Turchi
同步语音翻译（SimulST）系统的目标是以尽可能低的延迟生成其输出，延迟通常根据平均延迟（AL）计算。在本文中，我们强调，尽管AL被广泛采用，但与相应的参考文献相比，AL为生成更长预测的系统提供了被低估的分数。我们还表明，这个问题具有实际意义，因为最近的SimulST系统确实有过度生成的趋势。作为一种解决方案，我们提出了LAAL（长度自适应平均滞后），一个修改后的版本的度量，考虑到过生成现象，并允许无偏评估下/过生成系统。
Simultaneous speech translation (SimulST) systems aim at generating their output with the lowest possible latency, which is normally computed in terms of Average Lagging (AL). In this paper we highlight that, despite its widespread adoption, AL provides underestimated scores for systems that generate longer predictions compared to the corresponding references. We also show that this problem has practical relevance, as recent SimulST systems have indeed a tendency to over-generate. As a solution, we propose LAAL (Length-Adaptive Average Lagging), a modified version of the metric that takes into account the over-generation phenomenon and allows for unbiased evaluation of both under-/over-generating systems.