III: Small: Optimization Techniques for Scalable Semantic Web Data Processing in the Cloud

使用基于云的数据处理平台是一种越来越有吸引力的大规模数据处理替代方案。对它们在大规模非结构化数据和结构化数据上的各种处理任务的使用进行了积极的调查。然而，由于许多社区越来越有兴趣使用语义Web技术在Web上实现更自动的数据共享和交换，因此非常大的、真实世界的语义Web数据集的可用性迅速激增。这类数据是半结构化的，并且比关系数据处理具有更复杂的处理要求，这是由于数据的细粒度建模以及在处理过程中还需要进行推理。因此，用于云数据处理平台的现有优化技术通常采用关系处理优化技术，不能解决此类工作负载的需求。此外，这种技术没有充分考虑诸如Hadoop的云运行时平台的细微差别，例如，作为成本度量的数据流长度，没有索引和统计的先验存在。该项目有助于深入了解Map Reduce平台上语义Web数据处理的查询优化需求。它的贡献包括一种新颖的嵌套TripleGroup数据模型和代数(NTGA)、代数和动态代价查询优化技术、工作间和内部共享技术、数据表示格式以及将语义Web优化技术集成到ApachePig等框架中的系统架构问题。这个项目的影响将横跨越来越多的社区，这些社区正在积极采用语义网原则，例如科学、商业、政府和其他通用社区。

The use of cloud-based data processing platforms is an increasingly attractive alternative for large-scale data processing. There is active investigation into their use for various types of processing tasks on large-scale unstructured and structured data. However, due to an increased interest in many communities to enable more automatic sharing and exchange of data on the Web using Semantic Web techniques, there is a rapid surge in the availability of very large, real-world, Semantic Web datasets. Such data are semi-structured and have more complex processing requirements than relational data processing due to the fine-grained modeling of data and also the need for inferencing during processing. Consequently, existing optimization techniques for cloud data processing platforms which often adapt relational processing optimization techniques do not address the needs of such workloads. Further, such techniques do not adequately account for the nuances of cloud runtime platforms such as Hadoop e.g., dataflow length as a cost metric, no a-priori existence of indexes and statistics. This project contributes insight into query optimization requirements for Semantic Web data processing on Map Reduce platforms. Its contributions include a novel Nested TripleGroup data model and Algebra (NTGA), algebraic and dynamic cost query optimization techniques; inter and intra-work sharing techniques, data representation formats and system architecture issues of integrating Semantic Web optimization techniques into frameworks such as Apache Pig. The impact of this project will cut across the increasing range of communities that are aggressively adopting Semantic Web tenets such as, scientific, business, government and other general-purpose communities.