CAREER: A Scalable, Declarative, Imprecise Database Management System

个人、公司、政府和科学家可获得的前所未有的数据量有望彻底改变娱乐、商业、治理和科学的运作方式。虽然数据便宜且丰富，但其中大部分数据的质量低于过去30年来管理的精确数据。构建处理这种不精确数据的应用程序是困难的：它要求开发人员处理标准数据管理挑战（例如，并发性和可伸缩性），同时处理不精确和不完整的数据，这通常使用统计或机器学习技术来完成（例如，插值和分类）。Hazy项目通过构建一个系统来解决这一挑战，该系统将关系数据库管理系统的范例与统计机器学习技术相集成。该项目进行了以下主要任务：（I）设计一种语言，将这些技术与标准SQL集成，（II）提出一种代数来实现这种语言，沿着支持自动优化（类似于标准RDBMS），以及（III）发现技术，以有效地维护统计模型，因为底层数据被更改或更新。最终目标是一个系统，使开发使用不精确数据的可伸缩应用程序变得与开发精确的应用程序一样容易。Hazy允许用户使用比以往更复杂的统计处理来处理更大量的数据。反过来，这使得新的应用在一个divese集的领域，如生命和物理科学传感应用，医疗保健和环境监测，基于企业和基于Web的信息extraction.The本项目的研究是用于开发的数据和基础设施，新的实践风格的课程，正在开发中的威斯康星大学麦迪逊分校。此外，这一基础设施将被用作外联工作的一部分，使高中生能够获得数据分析工具。Hazy的源代码已开放，其结果在项目网站上公布（http：//www.cs.wisc.edu/hazy/）。

The unprecedented amounts of data available to individuals, companies, governments, and scientists promises to revolutionize the way entertainment, business, governance, and science operate. And while data are cheap and plentiful, much of this data is lower quality than the precise data that has been managed for the last 30 years. Building an application that processes this imprecise data is difficult: it requires that developers handle both standard data management challenges (e.g., concurrency and scalability), while at the same time coping with imprecise and incomplete data, which is typically done using statistical or machine learning techniques (e.g., interpolation and classification). The Hazy project addresses this challenge by building a system that integrates the paradigms of relational database management systems with statistical machine learning techniques. This project conducts the following major tasks: (I) designing a language to integrate these techniques with standard SQL, (II) proposing an algebra to implement this language along with support for automatic optimization (similar to a standard RDBMS), and (III) discovering techniques to efficiently maintain the statistical models as the underlying data are changed or updated. The end goal is a system that makes it as easy to develop scalable applications that use imprecise data as it is to develop their precise counterparts. Hazy allows users to process larger amounts of data with more sophisticated statistical processing than ever before. In turn, this enables new applications in a divese set of areas, such as life and physical science sensing applications, health-care and environmental monitoring, and enterprise-based and Web-based information extraction.The research of this project is used to develop the data and infrastructure for new practicum-style courses that are under development at the University of Wisconsin-Madison. In addition, this infrastructure will be used as part of an outreach effort to enable high school students to gain access to data analysis tools. The source code of Hazy is released into open source and the results are disseminated on the project Web site (http://www.cs.wisc.edu/hazy/).