What to Learn, and How: Toward Effective Learning from Rationales
What to Learn, and How: Toward Effective Learning from Rationales
复制标题
DOI：
10.18653/v1/2022.findings-acl.86
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
Samuel Carton;Surya Kanoria;Chenhao Tan
中科院分区：
文献类型：
--
作者：
Samuel Carton;Surya Kanoria;Chenhao Tan
从基本原理中学习试图使用人类注释的基本原理（即输入标记的子集）来提高模型预测的准确性，这些基本原理通常以中间或多任务监督的形式证明其选择的标签。虽然直观，但这个想法在实践中被证明是难以捉摸的。通过实证分析，我们发现：1）最大化理性监督的准确性并不一定是提高模型准确性的最佳目标; 2）人类理性的差异在于它们是否为模型提供了足够的预测信息。基于这些见解，我们提出了几种新的损失函数和学习策略，并在三个具有人类理性的数据集上评估了它们的有效性。我们的研究结果表明，标签和原理准确性均比基线有一致的改善，包括MultiRC的准确性提高了3%。我们的工作强调了理解人类解释的属性并在模型训练中相应地利用它们的重要性。
Learning from rationales seeks to augment model prediction accuracy using human-annotated rationales (i.e. subsets of input tokens) that justify their chosen labels, often in the form of intermediate or multitask supervision. While intuitive, this idea has proven elusive in practice. We make two observations about human rationales via empirical analyses:1) maximizing rationale supervision accuracy is not necessarily the optimal objective for improving model accuracy; 2) human rationales vary in whether they provide sufficient information for the model to exploit for prediction.Building on these insights, we propose several novel loss functions and learning strategies, and evaluate their effectiveness on three datasets with human rationales. Our results demonstrate consistent improvements over baselines in both label and rationale accuracy, including a 3% accuracy improvement on MultiRC. Our work highlights the importance of understanding properties of human explanations and exploiting them accordingly in model training.