Natural Language Data Management

点击翻译按钮获取中文摘要

A large volume of data generated everyday is in some form of natural language intended for human consumption; this includes news articles, blog posts, tweets, scientific articles, Wikipedia entries, financial reports, etc. However, our querying capabilities over this data have remained very much limited to keyword search, which reduces the efficiency of the search and the scope of the information that can be retrieved. Specifically, a keyword search is not sufficient when the search is not limited to selection and involves join and set operations over the contents of the documents. Additionally, a keyword search is not very applicable when the granularity of the search result is smaller than a document.This proposal advances the research in querying natural language data and the study of issues that hinder querying and managing such data (including both structured and unstructured) in documents. The particular challenges to be studied are: (1) storage and indexing, (2) querying and query processing, and (3) data integration and aggregation.(1) Standard text-based indices such as inverted index often ignore the structure of natural language data and will not provide the best support for queries. A storage system for natural language data may track both the ordering and the lexical relations between words and between senses to better support certain classes of queries. For example, synonymy and hyponymy relationships may indicate a degree of locality in the sense that a document that matches a word is likely to match the synonyms and hyponyms of the word as well.(2) Natural language data may be stored in and queried using a relational database, but composing queries over data can be cumbersome and relational systems may not provide the best support for the queries. A natural language data management system is expected to be geared towards the needs of applications that use natural language data by providing native support and treating natural language data as first class citizens. In particular, natural language data may be transformed to a meaning representation to better support reasoning and entailment detection and for integration with other sources. The querying system can then provide some support for these transformations; the querying system can also utilize the known relationships between fragments (e.g. distributional similarity) in both evaluating the queries and optimizing their evaluation.(3) Natural language data that resides in different sources can refer to the same entities differently; even the references within the same source can be ambiguous if taken out of their contexts. Ambiguities introduce problems in integrating and aggregating data from multiple sources. Despite the progress in the area of entity resolution, many challenges remain. We will work toward addressing the challenges related to querying, by exploiting new developments in knowledge bases and linked data.