Procella: Unifying serving and analytical data at YouTube
Procella: Unifying serving and analytical data at YouTube
复制标题
DOI：
10.14778/3352063.3352121
复制
发表时间：
2019-08
期刊：
影响因子：
--
通讯作者：
Biswapesh Chattopadhyay;Priyam Dutta;Weiran Liu;Ott Tinn;Andrew McCormick;Aniket Mokashi;Paul Harvey;Hector Gonzalez;David Lomax;Sagar Mittal;Roee Ebenstein;Nikita Mikhaylin;Hung-Ching Lee;Xiaoyan Zhao;Tony Xu;Luis Perez;Farhad Shahmohammadi;Tran Bui;Neil Mckay;Selcuk Aya;Vera Lychagina;Brett Elliott
中科院分区：
文献类型：
--
作者：
Biswapesh Chattopadhyay;Priyam Dutta;Weiran Liu;Ott Tinn;Andrew McCormick;Aniket Mokashi;Paul Harvey;Hector Gonzalez;David Lomax;Sagar Mittal;Roee Ebenstein;Nikita Mikhaylin;Hung-Ching Lee;Xiaoyan Zhao;Tony Xu;Luis Perez;Farhad Shahmohammadi;Tran Bui;Neil Mckay;Selcuk Aya;Vera Lychagina;Brett Elliott
像YouTube这样的大型组织正在处理爆炸式的数据量和对数据驱动应用程序日益增长的需求。从广义上讲，这些可以分类为：报告和指示板、页面中嵌入的统计信息、时间序列监视和特别分析。通常，组织会为每一个用例构建专门的基础结构。然而，这会造成数据和处理的竖井，并导致复杂、昂贵且难以维护的基础设施。在YouTube，我们通过构建一个新的SQL查询引擎Procella解决了这个问题。Procella在一个产品中实现了一个功能的超集，以高规模和高性能解决上述所有四个用例。今天，Procella每天在YouTube和其他几个bb0产品领域的所有四种工作负载上处理数千亿次查询。
Large organizations like YouTube are dealing with exploding data volume and increasing demand for data driven applications. Broadly, these can be categorized as: reporting and dashboarding, embedded statistics in pages, time-series monitoring, and ad-hoc analysis. Typically, organizations build specialized infrastructure for each of these use cases. This, however, creates silos of data and processing, and results in a complex, expensive, and harder to maintain infrastructure. At YouTube, we solved this problem by building a new SQL query engine - Procella. Procella implements a superset of capabilities required to address all of the four use cases above, with high scale and performance, in a single product. Today, Procella serves hundreds of billions of queries per day across all four workloads at YouTube and several other Google product areas.