III: Small: Managing Large-scale Uncertain Data Repositories

点击翻译按钮获取中文摘要

Increasing numbers of real-world application domains are generating data that isinherently noisy, incomplete, and probabilistic in nature. Examples of suchdata include measurement data collected by sensor networks, observation data inthe context of social networks, scientific and biomedical data, and datacollected by various online cyber-sources. The data uncertainties may be aresult of the fundamental limitations of the underlying measurementinfrastructures, the inherent ambiguity in the domain, or they may be aside-effect of the rich probabilistic modeling typically performed to extracthigh-level events from sensor and cyber data. Similarly, when attempting tointegrate heterogeneous data sources ("data integration") or extractingstructured information from text ("information extraction"), the results areapproximate and uncertain at best. However, there is currently a lack of datamanagement tools that can reason about large volumes of uncertain data, andhence the information about the uncertainty is often either discarded orreasoned about only superficially.In this project, we are building a complete probabilistic data managementsystem, called PrDB, that can manage, store, and process large-scalerepositories of uncertain data. PrDB unifies ideas from "large-scale structuredgraphical models" like probabilistic relational models (PRMs), developed in themachine learning literature, and "probabilistic query processing", studied inthe database literature. PrDB framework is based on the notion of "sharedfactors", which not only allows us to express and manipulate uncertainties atvarious levels of abstractions, but also supports capturing rich correlationsamong the uncertain data. PrDB supports a declarative SQL-like language forspecifying uncertain data and the correlations among them. PrDB also supportsexact and approximate evaluation of a wide range of queries including inferencequeries, SQL queries, and decision-support queries.The cross-disciplinary research undertaken during this project will enable us tosimultaneously address the challenges in the areas of probabilistic databasesand machine learning, and allow us to transfer the key technologies developedbetween those areas, thus advancing the research in both areas. It will enablethe development of a significant and high-impact new class of real-worldapplications, in a variety of domains including health informatics, social mediamanagement, World Wide Web, and scientific databases. The PrDB system sourcecode, and the datasets generated during the project, will be released using anappropriate open source license, at the project web site:http://www.cs.umd.edu/db/PrDB.html