III:Small:Enabling Technology for Best-Effort Data Integration Systems

在过去的几十年里，数据集成（DI）问题受到了极大的关注。 最初的大部分注意力都集中在集成业务数据上。这样的数据通常需要精确的集成，因为任何更少的东西显然都是不可用的。今天，这种精确的DI系统继续发挥重要作用。但它们不适合许多新兴领域，如个人信息管理、构建Web社区门户、科学数据管理、商业智能文本管理、公共安全和军事情报分析。第一，它们通常是“一次性”建造的，因为系统基本上是不可用的，直到它以其所有设想的通用性完成。第二，当出现新数据时，这些系统在向用户提供数据之前往往会出现长时间的延迟。第三，它们通常不是为了从用户反馈中获益而设计的，尽管在今天的Web 2.0世界中，这种反馈的机会经常存在。第四，精确DI系统在向用户解释答案方面提供很少或没有帮助。 作为回应，这个项目探索了一种范式转变，从精确的DI系统到尽力而为的系统。这些系统不是一次性构建的，而是逐步构建的。 他们的数据总是以某种方式可查询的。他们容忍数据中的错误，并可以利用用户反馈来随着时间的推移进行改进。最后，他们可以向用户解释他们的答案，从而使他们能够理解，验证和信任查询结果。（1）通过声明性的、语义透明的提取/集成语言的规范和实现，以及有效的优化和执行框架，增加对增量开发的支持。(2)通过设计和实施技术来利用用户社区的力量，这些技术允许用户在提取/集成过程中遇到错误时进行纠正，在整个提取和集成数据中始终传播这些纠正，并使用这些纠正来提高提取/集成模块的质量。(3)开发和实现捕获信息的技术，这些信息将帮助用户对系统数据进行沿着，并支持探索这些信息的含义。 该团队将联合收割机的技术，建立一个原型端到端的最佳努力DI系统和评估系统的三个现实世界的应用程序：DBLife门户网站，GLEON湖沼学项目，和madison.com Web portal.This研究将结合在正在进行的努力，在教育学生的技术提取和集成结构化数据。 将继续将代表性不足的少数群体纳入项目。 这一项目的成果将纳入2010-2011年出版的数据整合教科书。该项目将促进数据整合系统的广泛部署，从而为社会带来更有效的信息管理和获取。它将在教育下一代专业工作者和研究人员方面发挥不可或缺的作用。 这项研究也将有助于在GLEON项目的背景下湖沼学领域的科学家。它也有可能帮助madison.com的开发人员构建一个对更大的麦迪逊社区有更大用处的系统。最后，该项目的数据和系统工件将在研究界广泛传播，以显著增强研究和教育的数据管理基础设施。

Over the past few decades, the problem of data integration (DI) has received significant attention. Much of the initial attention was directed at integrating business data. Such data typically requires exact integration, because anything less is clearly not usable. Today, such exact DI systems continue to play an important role. But they are ill-suited for many emerging domains, such as personal information management, building Web community portals, scientific data management, text management for business intelligence, public safety, and military intelligence analysis. First, they are typically constructed in ``one shot'' in that the system is substantially unusable until it is completed in all of its envisioned generality. Second, when presented with new data, these systems often incur long delays before making the data available to users. Third, they typically are not designed to benefit from user feedback, even though opportunities for such feedback often exist in today's Web 2.0 world. Fourth, exact DI systems provide little or no assistance in explaining answers to users. In response, this project explores a paradigm shift, from precise DI systems to best-effort ones. Instead of being constructed in one shot, these systems are constructed incrementally. Their data is always queryable some fashion. They tolerate mistakes in the data, and can leverage user feedback to improve over time. Finally, they can explain their answers to the users, thereby allowing them to understand, verify, and trust query results.To build best-effort DI systems, researchers will pursue the following technical thrusts. (1)Increasing support for incremental development through the specification and implementation of a declarative, semantically transparent extraction/integration language, together with an effective optimization and execution framework. (2) Leveraging the power of a user community through the design and implementation of techniques that allow users to correct errors in the extraction/integration process as they are encountered, that consistently propagates these corrections throughout the extracted and integrated data, and that use these corrections to improve the quality of extraction/integration modules. (3) Developing and implementing techniques to capture information that will help users reason about the system's data along with support for exploring the implications of this information. The team will combine the technology to build a prototype end-to-end best-effort DI system and evaluate the system on three real-world applications: the DBLife portal, the GLEON limnology project, and the madison.com Web portal.This research will be integrated with ongoing efforts in educating students on techniques for extracting and integrating structured data. Inclusion of underrepresented minorities in the projects will be continued. The results from this project will be incorporated into a textbook on data integration to be published in 2010-2011. The project will facilitate the widespread deployment of data integration systems, thus resulting in more effective information management and access for society. It will play an integral part in educating next-generation professional workers and researchers. The research will also help domain scientists in limnology in the context of the GLEON project. It also has the potential to help the developers of madison.com build a system of much greater use to the greater Madison community. Finally, data and system artifacts from the project will be disseminated broadly in the research community to significantly enhance the data management infrastructure for research and education.