Improving Big Data Infrastructure to Effectively Utilize Resources for Data Processing and Services

点击翻译按钮获取中文摘要

Big data analytics is hailed as an emerging driving force of the next innovation wave. It has formed the basis for many novel applications across a wide range of fields from business workflows to engineering activities and scientific computing. One such typical application is cloud-based network services for business and individual mobile or vehicular users, where data is collected through daily log files, from stationary or mobile entities such as smartphones equipped with a variety of built-in sensors (e.g., GPS, accelerometers and cameras) and then processed by a big-data infrastructure in order to provision data as a service. For example, vehicle trajectories can be generated in real time from video data to facilitate the analysis of traffic flow parameters for transportation management. Clearly, in such applications, the response time of the underlying infrastructure for data processing has great impact on Quality of Service (QoS). *******My ongoing overall research goal is the amelioration of QoS through better resource utilization in big data infrastructure. I approach this research goal from two directions: First, I aim at better balancing network traffic and more efficiently sharing cloud resources for applications by exploiting network topology, job characteristics and execution processes. These proposed improvements are particularly important for service providers deploying big data infrastructure in clouds, as the provisioned resources are typically based on pay-as-you-go billing strategies, and effectively utilizing the resources means maximizing the monetary profits for cloud service providers and minimizing cost for users of big data analytics. Second, I search for an effective way to disseminate data to the outside world as a service. Ultimately, I want to increase QoS while also improving the tolerance of big data infrastructure for a variety of service-disrupting faults. ******According to a recent whitepaper on the Big Data market written by International Data Corporation and sponsored by SAS, a leading developer of analytics software, "Canadian businesses are at risk of falling behind international competitors in the era of Big Data". In particular, this survey and analysis found that Canadian companies are late and slow in adoption of technology capable of processing Big Data in comparison to international companies that are more willing to invest in and have more defined plans for adoption of these technologies in the near future.******My proposed research will provide a more cost effective and more reliable Big Data infrastructure that offers faster processing of Big Data. This will attract more Canadian companies to adopt this new technology in order to become more innovative and productive. My results will bring significant economic benefits to Canada's telecommunication and automobile industries, in particular, and significantly enhance Canada's profile in this field.