EAGER: Exploratory Evaluation: Scalability and Effectiveness of Data-Intensive Table-based Computing Software Systems

今天的科学和分析越来越多地通过对高分辨率捕获或模拟数据的系统探索来解决。通过更广泛的原始数据样本，可以形成更详细的模型和对数据含义的精确问题。 然而，用于处理海量数据集的大规模并行系统使得传统的编程、存储和容错策略无效。 基于表或面向列的分布式数据存储系统正在开发，以支持这种大规模的数据分析，由谷歌？的BigTable，并包括开源变体，如Apache Hbase。 这些新系统具有数据库行和列组织的风格，但具有更简单的语义、更弱的隔离性和非SQL接口。这些新系统对于互联网搜索支持以外的应用程序的有效性还没有得到很好的理解。这个探索性的项目正在开发一个评估框架，并探索一套新的基于表的存储系统，目的是了解最新技术水平，它们如何执行和扩展，以及它们的可靠性和可用性。除了关注关键指标的基准测试外，该项目的评估框架包括真实的世界应用程序，这些应用程序来自机器学习方法，用于理解事件流，如互联网博客出版物，以及理解复杂相互关系的方法，如社交网络图，以便深入了解启用这些新兴类型的知识发现应用程序所需的要求。

Science and analysis today are increasingly tackled by systematic exploration of high-resolution captured, or simulated, data. With a more expansive sample of raw data, more detailed models and precise questions of the meaning of the data can be formed. However, massively parallel systems for processing massive data sets render traditional programming, storage and fault tolerance strategies ineffective. Table-based or column-oriented distributed data storage systems are being developed to support such large scale data analysis, led by Google?s BigTable and including open source variations such as Apache Hbase. These new systems have the flavor of database row and column organization, but have simpler semantics, weaker isolation, and non-SQL interfaces, for example. The effectiveness of these new systems for applications other than internet search support is not well understood.This exploratory project is developing an evaluation framework and exploring a set of these new table-based storage systems, with the goal of capturing an understanding of the state of the art, how they perform and scale, and their reliability and usability.In addition to benchmarks focussing on key metrics, the project's evaluation framework includes real world applications drawn from machine learning approaches to understanding streams of events, such as internet blog publications, and approaches to understanding complex interrelationships, such as social networking graphs, in order to extract insight about the requirements needed to enable these emerging types of knowledge discovery applications.