Natural Language Data Management

每天产生的大量数据是以某种形式的自然语言供人类消费的；这包括新闻文章、博客文章、推文、科学文章、维基百科条目、金融报告等。然而，我们对这些数据的查询能力仍然非常局限于关键字搜索，这降低了搜索的效率和可检索信息的范围。具体地说，当搜索不限于选择并且涉及对文档内容的联接和集合操作时，关键字搜索是不够的。此外，当搜索结果的粒度小于文档时，关键字搜索不太适用。 这一建议推动了自然语言数据查询的研究，以及阻碍查询和管理文档中的自然语言数据(包括结构化和非结构化数据)的问题的研究。需要研究的特殊挑战是：(1)存储和索引，(2)查询和查询处理，以及(3)数据集成和聚合。 (1)标准的基于文本的索引，如倒排索引，往往忽略了自然语言数据的结构，不能为查询提供最佳的支持。自然语言数据的存储系统可以跟踪单词之间和词义之间的排序和词汇关系，以更好地支持某些类别的查询。例如，同义词和下义词关系可以在某种意义上指示位置的程度，即匹配单词的文档也可能匹配该单词的同义词和下义词。 (2)自然语言数据可以存储在关系数据库中并使用关系数据库进行查询，但在数据上编写查询可能会很麻烦，并且关系系统可能不会为查询提供最佳支持。自然语言数据管理系统应面向使用自然语言数据的应用程序的需要，提供本地支持，并将自然语言数据视为一等公民。具体地说，自然语言数据可以被转换成意义表示，以更好地支持推理和蕴涵检测，并用于与其他来源的集成。查询系统然后可以为这些转换提供一些支持；查询系统还可以在评估查询和优化其评估两者中利用片段之间的已知关系(例如，分布相似性)。 (3)不同来源的自然语言数据可以不同地引用相同的实体；如果脱离上下文，即使是同一来源中的引用也可能是模棱两可的。歧义给集成和聚合来自多个来源的数据带来了问题。尽管在实体解决领域取得了进展，但仍然存在许多挑战。我们将通过利用知识库和关联数据的新发展，努力解决与查询相关的挑战。

A large volume of data generated everyday is in some form of natural language intended for human consumption; this includes news articles, blog posts, tweets, scientific articles, Wikipedia entries, financial reports, etc. However, our querying capabilities over this data have remained very much limited to keyword search, which reduces the efficiency of the search and the scope of the information that can be retrieved. Specifically, a keyword search is not sufficient when the search is not limited to selection and involves join and set operations over the contents of the documents. Additionally, a keyword search is not very applicable when the granularity of the search result is smaller than a document. This proposal advances the research in querying natural language data and the study of issues that hinder querying and managing such data (including both structured and unstructured) in documents. The particular challenges to be studied are: (1) storage and indexing, (2) querying and query processing, and (3) data integration and aggregation. (1) Standard text-based indices such as inverted index often ignore the structure of natural language data and will not provide the best support for queries. A storage system for natural language data may track both the ordering and the lexical relations between words and between senses to better support certain classes of queries. For example, synonymy and hyponymy relationships may indicate a degree of locality in the sense that a document that matches a word is likely to match the synonyms and hyponyms of the word as well. (2) Natural language data may be stored in and queried using a relational database, but composing queries over data can be cumbersome and relational systems may not provide the best support for the queries. A natural language data management system is expected to be geared towards the needs of applications that use natural language data by providing native support and treating natural language data as first class citizens. In particular, natural language data may be transformed to a meaning representation to better support reasoning and entailment detection and for integration with other sources. The querying system can then provide some support for these transformations; the querying system can also utilize the known relationships between fragments (e.g. distributional similarity) in both evaluating the queries and optimizing their evaluation. (3) Natural language data that resides in different sources can refer to the same entities differently; even the references within the same source can be ambiguous if taken out of their contexts. Ambiguities introduce problems in integrating and aggregating data from multiple sources. Despite the progress in the area of entity resolution, many challenges remain. We will work toward addressing the challenges related to querying, by exploiting new developments in knowledge bases and linked data.