Design and implementation of big complex semantic data management system

点击翻译按钮获取中文摘要

In the age of big data, how to effectively store, manage, retrieve, and analyze such large-scale complex data in a timely and efficient manner is a major challenge. The primary goal of the proposed research is to design and implement a cost effective and highly scalable database management system for big complex semantic data storage, query, and analytics by combining and extending ideas from parallel database systems, MapReduce computing platform, and our previous work on complex semantic database model Information Networking Model (INM) and INM-DBMS. A parallel database system is a database management system (DBMS) implemented on a multiprocessor system with high-degree connectivity. It features data modeling using well-defined schemas, declarative query languages with high levels of abstraction, sophisticated query optimizers, and a rich runtime environment that supports efficient execution strategies. MapReduce computing paradigm, started by Google and made popular by the open source Hadoop, is a cost-effective distributed data storage and processing systems on large clusters of low-cost commodity machines connected with high-bandwidth network. It has gained a lot of attention in recent years from industry and research. INM-DBMS is a complex semantic database management system that features hierarchical and composite binary and higher degree relationships and their combinations, built-in semantics for consistency and integrity constraints for various relationships, and rich deductive and active rules. It has a concise and compact but expressive language consisting of three parts, information definition language (IDL), information manipulation language (IML) and information query language (IQL). IDL and IML provide powerful constructs to express the rich semantics and integrity constraints associated with various relationships. The declarative query language IQL effectively incorporates many useful features found in database, XML and logic programming languages such as logical variables, implicit existential and non-existential quantification, explicit universal quantification, negation as failure, tree expressions, etc. It can explore the natural networking structure of objects to extract and construct meaningful results in a concise, natural, and compact way. We will first extend the INM data model to adapt to schema-free and semi-structured, and then implement a big semantic database management system based on it. The system will consist of two layers to achieve high concurrency. The manipulation layer is in charge of definitions, manipulations and queries while Data Layer takes care of data storage. In the manipulation layer, objects are evenly allocate to storage nodes by a hash function and query tasks are analysed and interpreted into parallel tasks via an INM MapReduce library interface to achieve partition balancing and partitioned parallelism. In the data layer, the structured semantic data is partitioned and stored in various nodes to ensure dynamic scalability. In the methodology part, four kinds of nodes are differentiated to execute different jobs. The nodes are independent of each other and separated from material machines, which makes the system highly extensible in physical resource allocation and extremely robust at fault-tolerance. Our developed big complex semantic DBMS can significantly contribute to the effective management, efficient retrieval, and timely analytics of various heterogeneous, semi-structured and unstructured massive data. It can be used in many applications such as semantic search engine, complex social network services with exploded increasing data, large-scale data analysis and data mining, knowledge graph establishing and knowledge discovery, etc.