A performance study of big data analytics platforms
A performance study of big data analytics platforms
复制标题
DOI：
10.1109/bigdata.2017.8258260
复制
发表时间：
2017-12
期刊：
影响因子：
--
通讯作者：
Pouria Pirzadeh;M. Carey;T. Westmann
中科院分区：
文献类型：
--
作者：
Pouria Pirzadeh;M. Carey;T. Westmann
大数据分析已经成为各种企业利用他们现在可以访问的大数据财富的宝贵工具。因此，不同类别的大数据系统中的各种解决方案正在出现，以满足他们的需求。在本文中，我们使用TPC-H基准测试来比较从大数据平台的主要类别中挑选的四个大数据系统的性能：商业并行关系数据库（来自传统DBMS世界），Hive和Spark SQL（来自SQL on Hadoop世界）以及AsterixDB（来自NoSQL系统世界）。所有这些系统都有足够丰富的查询API和运行时系统来运行完整的TPC-H。另一方面，这些系统在体系结构、首选存储格式、对复杂模式定义的支持以及查询处理方法方面也有很大的差异。这使得它们成为一组非常有趣的代表性大数据系统进行比较。我们提出的结果，我们通过运行这些系统在不同的TPC-H规模使用各种设置，我们更详细地分析了一组选定的有趣的查询结果，探索性能，存储格式和模式定义之间的权衡。还包括后续讨论，以总结从这一努力中吸取的经验教训。
Big Data analytics has become an invaluable tool in a wide variety of businesses for exploiting the wealth of Big Data that they now have access to. As a result, various solutions within different categories of Big Data systems are emerging to meet their needs. In this paper we use the TPC-H benchmark to compare the performance of four Big Data systems picked from the major categories of Big Data platforms: a commercial parallel relational database (from the traditional DBMS world), Hive and Spark SQL (from the SQL-on-Hadoop world), and AsterixDB (from the world of NoSQL systems). All of these systems have sufficiently rich query APIs and runtime systems to run TPC-H in its full form. On the other hand, the systems also have major differences in terms of their architectures, preferred storage formats, support for complex schema definitions, and approaches to query processing. This makes them a very interesting set of representative Big Data systems to compare. We present the results that we obtained through running these systems at different TPC-H scales using various settings, and we analyze a selected set of interesting query results in more detail to explore the trade-offs between performance, storage formats, and schema definitions. A follow-up discussion is included as well to summarize the lessons learned from this effort.