RI: Small: Fast, Scalable Joint Inference for NLP using Markov Logic

自然语言处理（NLP）中的许多基本任务都涉及复杂的输出约束，如共指解析和事件提取。马尔可夫逻辑网络（mln）是一种结合逻辑和概率表示的联合推理框架，能够以紧凑的方式手动规范这些约束，有效地允许将背景知识轻松整合到NLP系统中以提高其性能。虽然理论上很有吸引力，但mln在NLP应用中的应用相对不足。由于可扩展性问题，研究人员在很大程度上局限于简单的mln，这些mln要么进行简化，有时是不合理的假设，要么忽略复杂的输出约束。该项目旨在为NLP中联合推理的应用方式带来革命性的变化。这个想法是为mln开发快速，可扩展的学习和推理技术，以便丰富的模型（即具有高维特征和/或复杂输出约束的模型）可以有效地训练并应用于大型数据集。该项目的一个关键组成部分是为重要和复杂的NLP任务（如共同参考分辨率）制定和评估丰富的基于mln的模型。这些丰富的模型，特别是在大型数据集上进行训练时，可能会在NLP中产生突破性的结果，从而产生深远的社会影响。例如，共同参考技术的改进基本上有利于公众每天依赖的所有NLP应用，例如搜索、信息提取和问题回答。将丰富的基于mln的模型成功应用于复杂的NLP任务需要开发快速、可扩展的学习和推理技术。为了扩大mln中的权重学习，该项目开发了利用约束满足文献中的高级算法进行快速、近似解计数的方法。为了扩大概率推理，它采用提升推理算法，通过利用精确和近似对称性（例如，释义）来减少mln中变量的域大小。它关注的核心NLP任务，如共参考解析和时间关系提取，足够复杂，它们为评估这些学习和推理技术的可扩展性提供了令人信服的测试平台。同样重要的是，由于近似语言是一种发生在NLP任务中的现象，这些进步可能会影响NLP中的大量任务。

Many fundamental tasks in natural language processing (NLP) such as coreference resolution and event extraction involve complex output constraints. Markov Logic Networks (MLNs), a joint inference framework that combines logical and probabilistic representations, enable manual specification of such constraints in a compact manner, effectively allowing easy incorporation of background knowledge into NLP systems to improve their performance. While theoretically appealing, MLNs have been relatively underused in NLP applications. Owing to issues of scalability, researchers have largely restricted themselves to simple MLNs that either make simplifying, sometimes unreasonable assumptions or ignore complex output constraints. This project seeks to bring transformative changes to the way joint inference is applied in NLP. The idea is to develop fast, scalable learning and inference techniques for MLNs so that rich models (i.e., models with high-dimensional features and/or complex output constraints) can be efficiently trained and applied to large data sets. A key component of the project is the formulation and evaluation of rich MLN-based models for important and complex NLP tasks such as coreference resolution. These rich models, especially when trained on large data sets, can potentially yield breakthrough results in NLP, which in turn can have profound societal impact. For example, improvements in coreference technologies stand to benefit essentially all NLP applications the general public relies on every day, such as search, information extraction, and question answering.Successful application of rich MLN-based models to complex NLP tasks requires the development of fast, scalable learning and inference techniques. To scale up weight learning in MLNs, this project develops approaches that leverage advanced algorithms from the constraint satisfaction literature for fast, approximate solution counting. To scale up probabilistic inference, it employs lifted inference algorithms to reduce the domain size of variables in MLNs by exploiting exact as well as approximate symmetries (e.g., paraphrases). The core NLP tasks it focuses on, such as coreference resolution and temporal relation extraction, are sufficiently complex that they provide convincing testbeds for evaluating the scalability of these learning and inference techniques. Equally importantly, as approximate language is a phenomenon that occurs across NLP tasks, these advances are likely to impact a wide swath of tasks in NLP.