Unanimous Prediction for 100% Precision with Application to Learning Semantic Mappings
Unanimous Prediction for 100% Precision with Application to Learning Semantic Mappings
复制标题
一致%20预测%20for%20100%%20精度%20和%20应用%20到%20学习%20语义%20映射
DOI：
10.18653/v1/p16-1090
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
Percy Liang
中科院分区：
文献类型：
--
作者：
Fereshte Khani;M. Rinard;Percy Liang
我们能否训练一个系统，使其对于任何新的输入，要么说“不知道”，要么做出保证正确的预测？如果我们的模型族定义明确，我们对这个问题的回答是肯定的。具体来说，我们引入一致性原则：只有当所有与训练数据一致的模型都预测出相同的输出时才进行预测。我们将这一原则应用于语义解析，即把话语映射到逻辑形式的任务。我们开发了一种简单、高效的方法，通过只检查两个模型来对所有一致模型的无限集合进行推理。我们证明，即使只有来自可能是对抗性分布的适量训练数据，我们的方法也能获得100%的准确率。在实验方面，我们在标准的GeoQuery数据集上证明了我们方法的有效性。
Can we train a system that, on any new input, either says "don't know" or makes a prediction that is guaranteed to be correct? We answer the question in the affirmative provided our model family is well-specified. Specifically, we introduce the unanimity principle: only predict when all models consistent with the training data predict the same output. We operationalize this principle for semantic parsing, the task of mapping utterances to logical forms. We develop a simple, efficient method that reasons over the infinite set of all consistent models by only checking two of the models. We prove that our method obtains 100% precision even with a modest amount of training data from a possibly adversarial distribution. Empirically, we demonstrate the effectiveness of our approach on the standard GeoQuery dataset.