Big data storage and retrieval technology for high-impact location derivative data

点击翻译按钮获取中文摘要

Evolution of computing and networking technology ushered in an era of "Big Data". 90% of the world's data has been produced in the last two years, and businesses are looking to leverage data to gain competitive edge. Massive volumes of data call for processing at massive scale. Old relational database (RDBMS) technology is often unable to support high query rates. RDBMS provides strong consistency guarantees and flexible querying at the expense of scalability; this motivates big data users to opt for NoSQL data stores. While NoSQL data stores generally scale, getting reasonable query response times can be challenging. The key is to design the data model in consideration of the query access pattern, applying so-called data denormalization techniques. This project will focus on automating the process of data denormalization: turning it from a time-consuming manual process to partly automated. We plan to develop tools that will analyze existing SQL queries and recommend denormalization heuristics for designers of NoSQL data models. We will address this problem in the context of Global Fleet Management: a Vancouver based company that performs GPS tracking of commercial vehicles. We expect that our experience will lead to development of generalized techniques for partly automated data denormalization.