Active Example Selection for In-Context Learning-猫眼课题宝

DOI：

10.48550/arxiv.2211.04486

发表时间：

2022-11

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Yiming Zhang;Shi Feng;Chenhao Tan

通过一些演示示例，大型语言模型展示了强大的能力，可以通过从这些示例中进行上下文学习来执行各种任务，而无需任何微调。我们证明，在上下文学习的表现可以是非常不稳定的样本的例子，表明语言模型如何获取信息的特质。我们制定的上下文学习的例子选择作为一个顺序决策问题，并提出了一个强化学习算法，用于确定可推广的政策，以选择示范的例子。对于GPT-2，我们学习的策略表现出很强的泛化能力，平均提高了5.8%。从我们学习的策略中选择的例子甚至可以在GPT-3 Ada上实现小的改进。然而，在更大的GPT-3模型上，这种改进会减少，这表明大型语言模型的新兴功能。

With a handful of demonstration examples, large-scale language models demonstrate strong capability to perform various tasks by in-context learning from these examples, without any fine-tuning. We demonstrate that in-context learning performance can be highly unstable across samples of examples, indicating the idiosyncrasies of how language models acquire information. We formulate example selection for in-context learning as a sequential decision problem, and propose a reinforcement learning algorithm for identifying generalizable policies to select demonstration examples. For GPT-2, our learned policies demonstrate strong abilities of generalizing to unseen tasks in training, with a 5.8% improvement on average. Examples selected from our learned policies can even achieve a small improvement on GPT-3 Ada. However, the improvement diminishes on larger GPT-3 models, suggesting emerging capabilities of large language models.

Active Example Selection for In-Context Learning

文献摘要

相似文献