Business Intelligence and Analytics in the Elastic Cloud

在过去的15年中，数据仓库和商业智能（BI/OLAP）应用程序已经成为企业决策支持系统的基石之一。最近，公共云计算模型在高性能BI领域提供了一个重要的替代方案。基于按需弹性的概念，云计算可用于创建其计算需求响应于工作负载变化或有利的定价机会而动态调整的应用程序。话虽如此，应用程序必须明确设计为识别和利用弹性。此外，为了在“按需付费”的公共云中具有成本效益，软件在执行过程中必须强调“收缩”周期和增长周期。目前的建议是建立在申请人的平行BI应用程序的经验。我们计划通过将并行算法和方法与列存储DBMS集成来利用这一专业知识，以实现动态、松散耦合的公共云。因为我们以前的工作广泛地利用了“突发”预聚合和批量更新周期，所以它几乎非常适合弹性平台。 具体而言，可以在动态实例化的计算集群上调用高度并行化的排序、聚合和合并过程，以便为最终的最终用户访问准备数据，从而极大地减少核心查询处理需求。设计和开发将在申请人现有的集群实现上进行，最终评估将在亚马逊的EC2云架构上进行。EC2不仅提供了广泛的可配置计算实例，还支持实例可访问的块存储卷、分布式缓存以及各种其他服务和选项。总之，我们相信并行算法和弹性的按需云服务的结合为提高BI应用程序的功能和成本效益提供了巨大的潜力。此外，分布式计算的一般收缩/增长模型应该作为跨各种领域的高性能云应用程序的模板。

Over the past 15 years data warehousing and Business Intelligence (BI/OLAP) applications have become one of the cornerstones of enterprise Decision Support systems. Recently, public cloud computing models have offered an important alternative in high performance BI domains. Based upon the notion of on-demand elasticity, cloud computing can be used to create applications whose computing requirements are dynamically adjusted in response to either workload variations or advantageous pricing opportunities. That being said, applications must be explicitly designed to identify and exploit elasticity. Moreover, in order to be cost effective within the "pay as you go" public cloud, software must emphasize the "shrink" cycle as much as the growth cycle during execution. The current proposal builds upon the applicant's experience with parallel BI applications. We plan to exploit this expertise by integrating parallel algorithms and methods with a column-store DBMS to target dynamic, loosely coupled, public cloud implementations. Because our previous work extensively utilizes "bursty" pre-aggregation and batch update cycles, it is almost ideally suited to elastic platforms. Specifically, heavily parallelized sorting, aggregation, and merging processes can be invoked on dynamically instantiated compute clusters in order to prepare data for eventual end user access, thereby dramatically minimizing core query processing requirements. Design and development will be carried out on the applicant's existing cluster implementation, with final evaluation conducted on Amazon's EC2 cloud architecture. EC2 not only provides a wide range of configurable compute instances, but also supports instance-accessible block storage volumes, distributed caching, and a variety of other services and options. In summary, we believe that the combination of parallel algorithms and elastic, on-demand cloud services provides tremendous potential for increasing both the power and cost-effectiveness of BI applications. Moreover, the general shrink/growth model of distributed computation should serve as template for high performance cloud applications across a variety of domains.