III: Small: Data Analysis in the Cloud with Guaranteed and Explainable Performance

越来越多的用户可以访问他们需要分析的大型数据集。天文学家、海洋学家和其他领域的科学家依靠数据分析来进行科学研究。记者可能希望分析数据以在文章中使用。在过去的几年中，云服务提供商已经为数据分析提供了越来越多的数据管理服务选择（例如，Amazon Elastic MapReduce或Google BigQuery）。云服务提供了对强大的数据分析工具的无缝访问，通常直接通过浏览器。然而，太多的服务仍然太接近于操作数据库管理系统的传统模式。它们揭示了太多关于其内部架构和部署的信息：用户需要在服务实例、实例类型和处理的GB级别进行推理。因此，今天的用户必须是数据管理专家，才能在这些服务之间进行选择，并以具有成本效益的方式利用它们。该项目将开发新的数据管理技术，使云服务提供商能够将用户与其服务内部的细节隔离开来，同时提供权衡价格和性能的能力。该项目将进一步开发工具来解释性能，并帮助用户重写他们的查询，以改善它。更具体地说，该项目将开发新的方法来（1）预测不仅查询运行时，但是否有可能执行慢于估计由于故障，偏斜，基数估计错误，或争用;（2）通过根据需要动态地改变分配给查询的资源及其故障处理和偏斜处理机制来保证查询运行时间;（3）在重负载的情况下公布特定的减速因子，并通过新颖的调度算法来保证它们;以及（4）以不需要用户理解查询计划的方式解释查询性能并建议重写。 该项目将在最近开发的开源Myria云数据管理系统（和服务）中实现所有算法，并在华盛顿大学持续运行。http://cloudperf.cs.washington.edu

Increasingly many users have access to large datasets that they need to analyze. Astronomers, oceanographers, and other domain scientists rely on data analysis for their science. Journalists may want to analyze data to use in their articles. Over the past several years, cloud service providers have been offering an increasingly large selection of data management services for data analytics (e.g., Amazon Elastic MapReduce or Google BigQuery). Cloud services provide a seamless access to powerful data analysis tools, often directly through the browser. Too many services, however, remain too close to the traditional mode of operating a database management system. They reveal too much information about their internal architecture and deployment: Users are required to reason at the level of service instances, instance types, and gigabytes processed. As a result, users today must be data management experts to choose between these services and leverage them in a cost-effective manner. This project will develop new data management techniques that will enable cloud service providers to isolate users from the details of their service internals while offering the ability to trade off price and performance. The project will further develop tools to explain performance and help users re-write their queries to improve it.More specifically, the project will develop new approaches to (1) predict not only the query runtime but whether a query is likely to execute slower than estimated due to failures, skew, cardinality estimation errors, or contention; (2) guarantee query runtimes by dynamically changing both the resources allocated to a query and its failure-handling and skew-handling mechanisms as needed; (3) post specific slowdown factors in case of heavy load and guarantee them through novel scheduling algorithms; and (4) explain query performance and suggest rewrites in a way that does not require users to understand query plans. The project will implement all of the algorithms in the open source Myria cloud data management system (and service) recently developed and in continuous operation at the University of Washington.For further information see the project web site at: http://cloudperf.cs.washington.edu