III-COR-Medium: Design and Implementation of the DBO Database System

商业和政府实体现在每年花费大约100亿美元用于管理“数据仓库”的软件和硬件系统，这些数据仓库是非常大的电子数据档案。尽管这个市场的规模和重要性，现有的数据管理解决方案可能是痛苦的缓慢（例如，见www.tpc.org/tpch/最近的基准测试结果）。现在，花费数百万美元的硬件和软件来建立一个系统，仍然需要几个小时来回答简单的分析问题。这是不幸的，因为有很多知识可以通过电子档案的交互式探索获得。很长的处理时间使数据很可能被存储起来，再也不会被查看。DBO数据库系统项目涉及到一个独特的系统的设计和开发，称为DBO。 像传统的关系数据库系统一样，DBO可以从头到尾运行数据库查询，并在非常大的档案中产生精确的答案。然而，与任何现有的研究或生产系统不同，DBO使用采样算法在整个查询执行过程中始终为最终查询答案产生统计估计。DBO给出的一个估算例子是，“真实答案在175万美元到180万美元之间的可能性为95%”。 用户等待的时间越长，估计就越准确。 这种估计的潜在好处是，当用户对估计的准确性不满意时，可以停止执行，这可能会在探索性处理期间节省大量时间。通过这种方式，DBO项目的目标是使交互式数据分析成为现实，即使是在最大的数据库上。该项目产生的所有科学和技术材料以及任何可供下载的软件都可以从http：//www.example.com获得。www.cise.ufl.edu/~cjermain/DBO

Commercial and government entities now spend around $10 billion peryear on software and hardware systems for managing "data warehouses",which are very large electronic data archives. Despite the size andimportance of this marketplace, existing data management solutions canbe painfully slow (see, for example, www.tpc.org/tpch/ for recentbenchmarking results). It is now possible to spend millions of dollarsof hardware and software for a system that still takes hours to answersimple analytic questions. This is unfortunate, because there is muchknowledge to be gained by interactive exploration of electronicarchives. Very long processing times make it likely that the data willstored away, and never looked at again.The DBO Database System project is concerned with the design anddevelopment of a unique system called DBO. Like traditionalrelational database systems, DBO can run database queries from startto finish and produce exact answers over very large archives. However,unlike any existing research or production system, DBO uses samplingalgorithms to produce a statistical estimate for the finalquery answer at all times throughout query execution. An example ofthe sort of estimate produced by DBO is, "There is a 95% chance that thetrue answer is between $1.75 million an $1.80 million". The longer auser waits, the more accurate the estimate becomes. The potentialbenefit of such an estimate is that a user can stop execution wheneversatisfied with the accuracy of the estimate, which may translate to dramatictime savings during exploratory processing. In this way, the goal ofthe DBO project is to render interactive data analysis a reality, evenover the largest databases.All scientific and technical materials produced by the project, aswell as any software available for download, can be obtained fromhttp://www.cise.ufl.edu/~cjermain/DBO .