Improving Big Data Infrastructure to Effectively Utilize Resources for Data Processing and Services

大数据分析被誉为下一次创新浪潮的新兴驱动力。它已经成为从业务工作流程到工程活动和科学计算等广泛领域的许多新应用程序的基础。一个这样的典型应用是用于商业和个人移动的或车辆用户的基于云的网络服务，其中通过日常日志文件从固定或移动的实体（诸如配备有各种内置传感器（例如，GPS、加速度计和摄像头），然后由大数据基础设施进行处理，以提供数据即服务。例如，可以从视频数据中真实的实时生成车辆轨迹，以便于分析交通流参数以进行运输管理。显然，在这样的应用程序中，底层基础设施的数据处理的响应时间对服务质量（QoS）有很大的影响。* 我正在进行的总体研究目标是通过更好地利用大数据基础设施中的资源来改善QoS。我从两个方向来实现这一研究目标：首先，我的目标是通过利用网络拓扑结构，作业特征和执行流程来更好地平衡网络流量，更有效地共享应用程序的云资源。这些建议的改进对于在云中部署大数据基础设施的服务提供商尤为重要，因为所提供的资源通常基于按需付费的计费策略，有效利用资源意味着最大限度地提高云服务提供商的货币利润，并最大限度地降低大数据分析用户的成本。第二，我寻找一种有效的方式来向外部世界传播数据作为一种服务。最终，我希望提高QoS，同时提高大数据基础设施对各种服务中断故障的容忍度。** 根据最近由国际数据公司撰写并由领先的分析软件开发商SAS赞助的大数据市场白皮书，“加拿大企业在大数据时代面临落后于国际竞争对手的风险”。特别是，这项调查和分析发现，与更愿意投资并在不久的将来采用这些技术的国际公司相比，加拿大公司采用能够处理大数据的技术的时间较晚，速度较慢。我提出的研究将提供一个更具成本效益和更可靠的大数据基础设施，提供更快的大数据处理。这将吸引更多的加拿大公司采用这项新技术，以提高创新性和生产力。我的研究结果将为加拿大的电信和汽车工业带来巨大的经济利益，特别是，并显着提高加拿大在这一领域的形象。

Big data analytics is hailed as an emerging driving force of the next innovation wave. It has formed the basis for many novel applications across a wide range of fields from business workflows to engineering activities and scientific computing. One such typical application is cloud-based network services for business and individual mobile or vehicular users, where data is collected through daily log files, from stationary or mobile entities such as smartphones equipped with a variety of built-in sensors (e.g., GPS, accelerometers and cameras) and then processed by a big-data infrastructure in order to provision data as a service. For example, vehicle trajectories can be generated in real time from video data to facilitate the analysis of traffic flow parameters for transportation management. Clearly, in such applications, the response time of the underlying infrastructure for data processing has great impact on Quality of Service (QoS). *******My ongoing overall research goal is the amelioration of QoS through better resource utilization in big data infrastructure. I approach this research goal from two directions: First, I aim at better balancing network traffic and more efficiently sharing cloud resources for applications by exploiting network topology, job characteristics and execution processes. These proposed improvements are particularly important for service providers deploying big data infrastructure in clouds, as the provisioned resources are typically based on pay-as-you-go billing strategies, and effectively utilizing the resources means maximizing the monetary profits for cloud service providers and minimizing cost for users of big data analytics. Second, I search for an effective way to disseminate data to the outside world as a service. Ultimately, I want to increase QoS while also improving the tolerance of big data infrastructure for a variety of service-disrupting faults. ******According to a recent whitepaper on the Big Data market written by International Data Corporation and sponsored by SAS, a leading developer of analytics software, "Canadian businesses are at risk of falling behind international competitors in the era of Big Data". In particular, this survey and analysis found that Canadian companies are late and slow in adoption of technology capable of processing Big Data in comparison to international companies that are more willing to invest in and have more defined plans for adoption of these technologies in the near future.******My proposed research will provide a more cost effective and more reliable Big Data infrastructure that offers faster processing of Big Data. This will attract more Canadian companies to adopt this new technology in order to become more innovative and productive. My results will bring significant economic benefits to Canada's telecommunication and automobile industries, in particular, and significantly enhance Canada's profile in this field.