Business Intelligence and Analytics in the Elastic Cloud

点击翻译按钮获取中文摘要

Over the past 15 years data warehousing and Business Intelligence (BI/OLAP) applications have become one of the cornerstones of enterprise Decision Support systems. Recently, public cloud computing models have offered an important alternative in high performance BI domains. Based upon the notion of on-demand elasticity, cloud computing can be used to create applications whose computing requirements are dynamically adjusted in response to either workload variations or advantageous pricing opportunities. That being said, applications must be explicitly designed to identify and exploit elasticity. Moreover, in order to be cost effective within the "pay as you go" public cloud, software must emphasize the "shrink" cycle as much as the growth cycle during execution. The current proposal builds upon the applicant's experience with parallel BI applications. We plan to exploit this expertise by integrating parallel algorithms and methods with a column-store DBMS to target dynamic, loosely coupled, public cloud implementations. Because our previous work extensively utilizes "bursty" pre-aggregation and batch update cycles, it is almost ideally suited to elastic platforms. Specifically, heavily parallelized sorting, aggregation, and merging processes can be invoked on dynamically instantiated compute clusters in order to prepare data for eventual end user access, thereby dramatically minimizing core query processing requirements. Design and development will be carried out on the applicant's existing cluster implementation, with final evaluation conducted on Amazon's EC2 cloud architecture. EC2 not only provides a wide range of configurable compute instances, but also supports instance-accessible block storage volumes, distributed caching, and a variety of other services and options. In summary, we believe that the combination of parallel algorithms and elastic, on-demand cloud services provides tremendous potential for increasing both the power and cost-effectiveness of BI applications. Moreover, the general shrink/growth model of distributed computation should serve as template for high performance cloud applications across a variety of domains.