Velox: Meta's Unified Execution Engine
Velox: Meta's Unified Execution Engine
复制标题
Velox：Meta 的统一执行引擎
DOI：
10.14778/3554821.3554829
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Biswapesh Chattopadhyay
中科院分区：
文献类型：
--
作者：
P. Pedreira;O. Erling;Masha Basmanova;Kevin Wilfong;Laith Sakka;K. Pai;Wei He;Biswapesh Chattopadhyay
针对非常特定的数据工作负载的新的专用计算引擎的特别开发已经创建了一个孤立的数据环境。通常，这些引擎彼此之间几乎没有共享，并且很难维护，发展和优化，最终为数据用户提供不一致的体验。为了解决这些问题，Meta创建了Velox，一个新颖的开源C++数据库加速库。Velox提供可重用、可扩展、高性能和方言无关的数据处理组件，用于构建执行引擎和增强数据管理系统。该库严重依赖于向量化和自适应性，并且由于其在现代工作负载中的普遍性，从根本上设计为支持复杂数据类型的高效计算。Velox目前已与Meta的十几个数据系统集成或正在集成，包括Presto和Spark等分析查询引擎、流处理平台、消息总线和数据仓库摄取基础设施、用于特征工程和数据预处理的机器学习系统（PyTorch）等。它提供了以下方面的好处：（a）通过民主化以前仅在单个引擎中发现的优化来提高效率，（B）提高数据用户的一致性，以及（c）通过促进可重用性来提高工程效率。
The ad-hoc development of new specialized computation engines targeted to very specific data workloads has created a siloed data landscape. Commonly, these engines share little to nothing with each other and are hard to maintain, evolve, and optimize, and ultimately provide an inconsistent experience to data users. In order to address these issues, Meta has created Velox, a novel open source C++ database acceleration library. Velox provides reusable, extensible, high-performance, and dialect-agnostic data processing components for building execution engines, and enhancing data management systems. The library heavily relies on vectorization and adaptivity, and is designed from the ground up to support efficient computation over complex data types due to their ubiquity in modern workloads. Velox is currently integrated or being integrated with more than a dozen data systems at Meta, including analytical query engines such as Presto and Spark, stream processing platforms, message buses and data warehouse ingestion infrastructure, machine learning systems for feature engineering and data preprocessing (PyTorch), and more. It provides benefits in terms of (a) efficiency wins by democratizing optimizations previously only found in individual engines, (b) increased consistency for data users, and (c) engineering efficiency by promoting reusability.
编译查询的自适应执行
DOI：
10.1109/icde.2018.00027
发表时间：
2018
期刊：
2018 IEEE 34th International Conference on Data Engineering (ICDE)
影响因子：
--
作者：
J. André Kohn;Viktor Leis;Thomas Neumann
通讯作者：
Thomas Neumann