I-Corps: Feasibility Study for Commercializing a Domain-Specific Big Data Analytics Cloud Software Stack

点击翻译按钮获取中文摘要

In the so-called big data era, the data in many companies are becoming too big to be processed and stored by traditional computing platforms. There are urgent needs to address the challenge from ever-increasing big data facing these industries, especially for small companies and contractors who do not have large scale computing infrastructure. Those big companies that have the large scale computing infrastructure, are currently using the traditional High Performance Computing (HPC) techniques to handle the big data requirements. However, the traditional HPC techniques were designed to process computation intensive works, instead of data-intensive jobs. Moreover, the traditional HPC requires deep knowledge and expertise in HPC architectures and uses low-level parallel programming models, which typically create an obstacle for the regular users and domain experts to utilize HPC infrastructure directly. This I-Corps team proposes the cloud computing technique that provides a cost-effective solution to address the computation and storage requirements, which is especially useful to small companies and startups.This project is to explore the commercialization opportunities for a software stack that stores, processes and analyzes big volume of data in a variety of domains, including petroleum seismic data, medical images, retails, energy and more. Besides the scalable performance and fault-tolerance features provided by the software stack, it also provides high-level programming templates, workflow and a domain specific language to simplify the usage of large scale computing platforms. The proposed cloud-based software stack will deliver a customized computing platform service via a user-friendly web interface that is designed to fill the gap between the big data analytics platforms and the customer?s requirements in a variety of domains. The targeted users are data scientists, analysts, researchers and developers in the petroleum, medical and other industrial domains dealing with big volume of data.