Fast Decision Support Queries in Main-Memory: Algorithms, Optimization and Implementation

组织可以将其数据放入集中式数据库中，从中可以进行复杂的决策支持查询。这些查询通常需要非常高效地处理和组合大量数据。这些查询的答案在应用程序领域可能非常有价值，可以对组织行为进行明智的更改。评估和优化一般决策支持查询的新技术正在主存数据库环境中开发。在RAM中存储千兆字节的数据已成为可能。因此，人们开始问决策支持查询处理是否可以获得将数据驻留在主存中的性能优势。所采用的方法是研究与在主存中处理复杂查询相关的性能问题。通过设计和使用专为主存数据库定制的算法、访问结构和数据表示，我们可以避免在基于磁盘的系统中明显存在的大部分开销，即使数据集适合主存。通过只允许在脱机批处理窗口中进行更新，可以获得组织数据的灵活性，从而实现快速查询响应。正在实现一个原型系统，以演示所建议的技术的好处，并根据查询响应时间进行度量。系统具有Web界面。原型机将使用哥伦比亚长老会医疗中心的医疗数据进行评估。在这个项目过程中开发的新技术将广泛应用于处理对数据集的复杂查询的应用程序，这些数据集可以放在主存储器中。http://www.cs.columbia.edu/~kar

An organization may place its data into a centralized database from which complex decision-support queries can be made. These queries typically need to process and combine large amounts of data very efficiently. Answers to these queries can be very valuable in the application domain, enabling informed changes to organizational behavior. New techniques for evaluating and optimizing general decision-support queries are being developed in the context of a main-memory database. It is becoming possible to store gigabytes of data in RAM. Thus one begins to ask whether decision support query processing could obtain the performance benefits of having data reside in main memory. The approach taken is to study the performance issues associated with processing complex queries in main memory. By designing and using algorithms, access structures, and data representations tailored to main-memory databases, we can avoid much of the overhead that is apparent in a disk-based system even when the data set fits in main-memory. By allowing updates only in an off-line batch window, one gains the flexibility to organize the data in a way that is tuned for fast query response. A prototype system is being implemented to demonstrate the benefits of the proposed techniques, measured in terms of query response times. The system has a Web interface. The prototype will be evaluated using medical data from the Columbia Presbyterian Medical Center. The novel techniques developed in the course of this project will have broad use in applications that process complex queries over datasets that can fit in main memory. http://www.cs.columbia.edu/~kar