Bandit Structured Prediction for Neural Sequence-to-Sequence Learning
Bandit Structured Prediction for Neural Sequence-to-Sequence Learning
复制标题
DOI：
10.18653/v1/p17-1138
复制
发表时间：
2017-04
期刊：
影响因子：
--
通讯作者：
Julia Kreutzer;Artem Sokolov;S. Riezler
中科院分区：
文献类型：
--
作者：
Julia Kreutzer;Artem Sokolov;S. Riezler
Bandit结构化预测描述了一种随机优化框架，其中学习是从部分反馈中进行的。该反馈以对预测输出结构的任务损失评估的形式接收，而无需访问金标准结构。我们通过使用基于注意的递归神经网络将线性强盗学习提升到神经序列到序列学习问题来推进该框架。此外，我们展示了如何将控制变量纳入我们的学习算法中，以减少方差和改进泛化。我们提出了对神经机器翻译任务的评估，该任务显示了从模拟强盗反馈中进行域适应的高达5.89 BLEU点的改进。
Bandit structured prediction describes a stochastic optimization framework where learning is performed from partial feedback. This feedback is received in the form of a task loss evaluation to a predicted output structure, without having access to gold standard structures. We advance this framework by lifting linear bandit learning to neural sequence-to-sequence learning problems using attention-based recurrent neural networks. Furthermore, we show how to incorporate control variates into our learning algorithms for variance reduction and improved generalization. We present an evaluation on a neural machine translation task that shows improvements of up to 5.89 BLEU points for domain adaptation from simulated bandit feedback.