III: Small: Optimization Techniques for Scalable Semantic Web Data Processing in the Cloud

III：小：云中可扩展语义Web数据处理的优化技术

• 批准号: 1218277
• 负责人: Kemafor Anyanwu-Ogan
• 金额: $ 44.69万
• 依托单位: North Carolina State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-09-01 至 2017-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1218277&HistoricalAwards=false
• 关键词: III; Small; Optimization; Techniques; Scalable

The use of cloud-based data processing platforms is an increasingly attractive alternative for large-scale data processing. There is active investigation into their use for various types of processing tasks on large-scale unstructured and structured data. However, due to an increased interest in many communities to enable more automatic sharing and exchange of data on the Web using Semantic Web techniques, there is a rapid surge in the availability of very large, real-world, Semantic Web datasets. Such data are semi-structured and have more complex processing requirements than relational data processing due to the fine-grained modeling of data and also the need for inferencing during processing. Consequently, existing optimization techniques for cloud data processing platforms which often adapt relational processing optimization techniques do not address the needs of such workloads. Further, such techniques do not adequately account for the nuances of cloud runtime platforms such as Hadoop e.g., dataflow length as a cost metric, no a-priori existence of indexes and statistics. This project contributes insight into query optimization requirements for Semantic Web data processing on Map Reduce platforms. Its contributions include a novel Nested TripleGroup data model and Algebra (NTGA), algebraic and dynamic cost query optimization techniques; inter and intra-work sharing techniques, data representation formats and system architecture issues of integrating Semantic Web optimization techniques into frameworks such as Apache Pig. The impact of this project will cut across the increasing range of communities that are aggressively adopting Semantic Web tenets such as, scientific, business, government and other general-purpose communities.

使用基于云的数据处理平台是大规模数据处理的一个越来越有吸引力的替代方案。人们正在积极研究它们在大规模非结构化和结构化数据的各种处理任务中的应用。然而，由于许多社区对使用语义Web技术在Web上实现更自动的数据共享和交换越来越感兴趣，因此非常大的、真实的语义Web数据集的可用性迅速增加。这类数据是半结构化的，由于需要对数据进行细粒度建模，并且在处理过程中需要进行推理，因此比关系数据处理具有更复杂的处理需求。因此，用于云数据处理平台的现有优化技术（通常采用关系处理优化技术）无法满足此类工作负载的需求。此外，这些技术不能充分考虑云运行时平台（如Hadoop）的细微差别，例如，作为成本度量的数据流长度，没有先验存在的索引和统计。该项目有助于深入了解Map Reduce平台上语义Web数据处理的查询优化需求。它的贡献包括一种新的嵌套TripleGroup数据模型和代数（NTGA），代数和动态成本查询优化技术；将语义Web优化技术集成到Apache Pig等框架中的工作间和内部共享技术、数据表示格式和系统架构问题。这个项目的影响将跨越越来越多的积极采用语义Web原则的社区，如科学、商业、政府和其他通用社区。