Time-Aware Language Models as Temporal Knowledge Bases
Time-Aware Language Models as Temporal Knowledge Bases
复制标题
作为时态知识库的时间感知语言模型
DOI：
--
复制
发表时间：
2021
影响因子：
10.9
通讯作者：
William W. Cohen
中科院分区：
文献类型：
--
作者：
Bhuwan Dhingra;Jeremy R. Cole;Julian Martin Eisenschlos;D. Gillick;Jacob Eisenstein;William W. Cohen
从总统的名字到篮球队的许多事实都有到期日期。 ，尤其是在封闭式的设置中，训练训练的语料库必须记住的事实，我们引入了旨在探究LMS的诊断数据集，以了解随着时间的流逝而变化并突出显示问题。在频谱的任何一端，都对临时数据的特定切片进行了训练，以及在各种临时数据上进行的训练改善了从训练时间期看到事实的记忆，以及对未来时间段的未见事实的校准。在新数据到达时有效地“刷新”，而无需从头开始重新审查。
Many facts come with an expiration date, from the name of the President to the basketball team Lebron James plays for. However, most language models (LMs) are trained on snapshots of data collected at a specific moment in time. This can limit their utility, especially in the closed-book setting where the pretraining corpus must contain the facts the model should memorize. We introduce a diagnostic dataset aimed at probing LMs for factual knowledge that changes over time and highlight problems with LMs at either end of the spectrum—those trained on specific slices of temporal data, as well as those trained on a wide range of temporal data. To mitigate these problems, we propose a simple technique for jointly modeling text with its timestamp. This improves memorization of seen facts from the training time period, as well as calibration on predictions about unseen facts from future time periods. We also show that models trained with temporal context can be efficiently “refreshed” as new data arrives, without the need for retraining from scratch.
影响因子：
22.7
作者：
Vrandecic, Denny;Kroetzsch, Markus
通讯作者：
Kroetzsch, Markus
DOI：
10.1162/tacl_a_00324
发表时间：
2020-01-01
影响因子：
10.9
作者：
Jiang, Zhengbao;Xu, Frank F.;Neubig, Graham
通讯作者：
Neubig, Graham
检查文档分类中的临时性
DOI：
--
发表时间：
2018
期刊：
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers
影响因子：
--
作者：
Huang, Xiaolei;Paul, Michael J.
通讯作者：
Paul, Michael J.