II: Information Integration in the Presence of Uncertainty

构建和部署数据集成系统的经验表明，它们在一个非常根本的方面是脆弱的：它们无法处理数据的不确定性，也无法处理数据如何组合以提供答案的不确定性。这一限制在科学应用中尤为明显，因为在科学应用中，数据本身是不确定的，领域的模型也在不断演变。从用户的角度来看，无法对不确定性进行建模可能会导致相关答案的丢失、无关答案的激增以及答案的不合理。这种局限性深深植根于当今数据管理系统中的确定性范式，该范式旨在支持对大型数据实例的可扩展性，但不能对不确定性进行表示和推理。在过去的几年里，已经有了生物介体系统，它集成了大约12个关于基因和蛋白质的公共数据源。该小组观察并记录了限制任何基于介体的集成系统(如BioMediator)的能力的不确定性类型。这些不确定性发生在三个级别：数据实例级别、模式级别和用户查询级别。在新的方法中，所有不确定性都将在系统中明确表示，并使用概率数据模型以统一的方式表示。该中介系统支持SQL查询语言，但具有修改的语义：每个查询的答案都用概率分数和线条信息标注。新的工作将涉及到概率数据模型的设计，概率查询处理和优化技术的发展，以及用户反馈方法的设计。他们将建立一个系统，U2(UII(不确定信息集成)的缩写)，该系统将对系统所有级别的不确定性进行建模，包括查询语言、中介模式、源映射和源数据。U2将向用户解释其结果，并将在出现不确定性时积极寻求解决，在可能的情况下纳入用户的反馈。他们将扩展BioMediator系统，并与该系统的当前用户合作。信息集成的问题将被更紧密地整合到本科生和研究生的数据库课程中。第二，这项研究将促进与生物医学计算研究的合作，并将扩展目前该领域从业人员使用的BioMediator系统。最后，将向公众提供工具和服务。

Experience building and fielding data integration systems has shown that they are brittle in a very fundamental way: they cannot handle uncertainty about data or about how data is combined to provide answers. This limitation is especially pronounced in scientific applications, where data is inherently uncertain and the models of the domain are constantly evolving. From the users' perspective, the inability to model uncertainty can result in loss of relevant answers, an explosion of irrelevant answers and in no justification of answers. The limitation is deeply rooted in the deterministic paradigm underpinning data management systems today, which is designed to support scalability to large data instances, but is incapable of representing and reasoning about uncertainty.A new approach to data integration, where uncertainties are handled explic-Itly, is proposed. Over the past few years, the BioMediator system, whichintegrates about a dozen public data sources on genes and proteins, has been available. The group has observed and documented the types of uncertainty that limit the power of any mediator-based integration system like BioMediator. These uncertainties occur at three levels: at the data instance level, at the schema level, and at the user query level. In the new approach, all uncertainties will be made explicit in the system, and represented in a uniform way, using a probabilistic data model. The mediator system supports a query language with SQL but with a modified semantics: the answers to each query are annotated with a probability score, and a lineageinformation.The new work will involve the design of a probabilistic data model, the development of probabilistic query processing and optimization techniques, and the design of user feedback methods. They will build a system, U2 (short for UII { UncertainInformation Integration ) that will model uncertainty at all levels of the system, including the query language, mediated schema, source mappings and source data. U2 will explain its results to the user and will actively seek to resolve uncertainty when it arises, incorporating feedback from the user where possible. They will extend the BioMediator System and collaborate with the current users of the system.There are three areas of broader impact. Issues of information integration will be integrated more tightly into the undergraduate and graduate database curriculumSecond, the research will fuel collaboration with biomedical computing research, and willextend the BioMediator system that is currently in use by practitioners in the field. Finally, tools and services will be made available for public use.