III-COR-Medium: Design and Implementation of the DBO Database System

商业和政府实体现在每年在管理“数据仓库”的软件和硬件系统上花费约100亿美元，“数据仓库”是非常大的电子数据档案。尽管这个市场的规模和重要性很大，但现有的数据管理解决方案可能非常缓慢(例如，有关最近的基准测试结果，请参见www.tpc.org/tpch/)。现在，花费数百万美元的硬件和软件来建造一个仍然需要几个小时来回答简单的分析问题的系统，现在是可能的。这是不幸的，因为通过对电子档案的互动探索可以获得许多知识。DBO数据库系统项目致力于设计和开发一种名为DBO的独特系统。与传统的关系数据库系统一样，DBO可以从头到尾运行数据库查询，并在非常大的归档中生成准确的答案。然而，与任何现有的研究或生产系统不同，DBO使用采样算法在整个查询执行过程中始终为最终查询答案生成统计估计。DBO做出的这种估计的一个例子是，“真正的答案有95%的可能性在175万美元到180万美元之间”。用户等待的时间越长，估计就越准确。这种估计的潜在好处是，只要用户对估计的准确性感到满意，就可以停止执行，这可以转化为在探索性处理期间显著节省时间。通过这种方式，DBO项目的目标是实现交互式数据分析，即使在最大的数据库上也是如此。该项目产生的所有科学和技术材料，以及任何可供下载的软件，都可以从http://www.cise.ufl.edu/~cjermain/DBO获得。

Commercial and government entities now spend around $10 billion per year on software and hardware systems for managing "data warehouses", which are very large electronic data archives. Despite the size and importance of this marketplace, existing data management solutions can be painfully slow (see, for example, www.tpc.org/tpch/ for recent benchmarking results). It is now possible to spend millions of dollars of hardware and software for a system that still takes hours to answer simple analytic questions. This is unfortunate, because there is much knowledge to be gained by interactive exploration of electronic archives. Very long processing times make it likely that the data will stored away, and never looked at again.The DBO Database System project is concerned with the design and development of a unique system called DBO. Like traditional relational database systems, DBO can run database queries from start to finish and produce exact answers over very large archives. However, unlike any existing research or production system, DBO uses sampling algorithms to produce a statistical estimate for the final query answer at all times throughout query execution. An example of the sort of estimate produced by DBO is, "There is a 95% chance that the true answer is between $1.75 million an $1.80 million". The longer a user waits, the more accurate the estimate becomes. The potential benefit of such an estimate is that a user can stop execution whenever satisfied with the accuracy of the estimate, which may translate to dramatic time savings during exploratory processing. In this way, the goal of the DBO project is to render interactive data analysis a reality, even over the largest databases.All scientific and technical materials produced by the project, as well as any software available for download, can be obtained from http://www.cise.ufl.edu/~cjermain/DBO .