BIGDATA: Collaborative Research: F: Holistic Optimization of Data-Driven Applications

我们每天都与网上购物和网上银行网站互动。许多这样的网站都是由数据驱动的应用程序驱动的。这样的应用程序通常由两部分组成：托管在应用服务器上的应用程序，以及托管在与维护持久数据的应用服务器分开的服务器上的数据库管理系统（DBMS）。不幸的是，许多数据驱动的应用程序都存在性能问题，例如加载页面需要很长时间，或者无法扩展以同时为大量客户机提供服务。在数据驱动的应用程序中发现和修复性能问题的最新技术是分别检查应用程序的两个部分，这样做会错过发现和修复此类问题的许多机会。与之前的方法不同，在本项目中，我们将同时处理DBMS和应用程序。特别是，我们将设计新的技术和工具来帮助识别性能问题，理解这些问题的原因，并自动修复它们。该项目将为跨层程序编译和优化开辟新的机会，其实际目标是提高数据驱动应用程序的性能，这将对我们日常生活的许多方面产生重大影响。这个项目的研究结果将被整合到芝加哥大学和华盛顿大学提供的本科和研究生软件工程、数据管理入门和编译器课程中。该项目的外展活动将包括通过针对代表性不足的群体的特殊项目，如由女性计算研究协会组织的本科生分布式研究经验（DREU）和由女性计算研究协会组织的多样性研讨会，吸引学生并为学生提供建议。具体来说，提出的研究包括三个重点：(1)一个新的跨层程序分析框架，该框架通过理解应用程序代码、应用程序发送给DBMS的查询以及DBMS如何处理这些查询来生成数据驱动应用程序的端到端概要；(2)一个程序分析和测试框架，通过利用从(1)中创建的端到端概要文件来识别数据驱动应用程序中的性能问题；(3)通过转换应用程序代码和发出的查询来优化数据驱动的应用程序的新方法。这三个重点将一起工作，以提高数据驱动应用程序的性能，并帮助程序员在开发过程中检测性能问题。本项目开发的软件、用于评估的基准以及与现有技术的性能比较将通过项目网站发布到公共领域。更多信息请访问项目网站（https://people.eecs.berkeley.edu/~akcheung/coopt.html）。

We interact with online shopping and banking websites on a daily basis. Many of these websites are powered by data-driven applications. Such application often consists of two parts: an application hosted on an application server, and a database management system (DBMS) hosted on a separate server from the application server that maintains persistent data. Unfortunately, many data-driven applications suffer from performance problems, such as taking a long time to load a page or inability to scale up to serve large number of clients simultaneously. The state of the art in discovering and fixing performance problems in data-driven applications is to examine the two parts of the application separately, and doing so misses many opportunities in discovering and fixing such problems. Unlike prior approaches, in this project we will treat the DBMS and the application in tandem. In particular, we will devise new techniques and tools to help identify performance problems, understand the cause of such problems, and fix them automatically. This project will open up new opportunities in cross-layer program compilation and optimization, with the practical goal of improving the performance of data-driven applications that will have a significant impact in many aspects of our daily lives. The findings from this project will be incorporated into undergraduate and graduate software engineering, introduction to data management, and compiler classes to be offered at the University of Chicago and the University of Washington. The outreach activities of this project will include engaging and advising students through special programs geared toward under-represented groups such as the Distributed Research Experiences for Undergraduates (DREU) organized by CRA-W (Computing Research Association -- Women) and Diversity Workshops organized by CRA-W.Specifically, the proposed research consists of three thrusts: (1) a new cross-layer program analysis framework that produces an end-to-end profile of data-driven applications by understanding the application code, the queries that the application sends to the DBMS, and how the DBMS processes such queries; (2) a program analysis and testing framework that identify performance problems in data-driven applications by leveraging the end-to-end profile created from (1); and (3) new means to optimize data-driven applications by transforming both the application code and the queries that are issued. These three thrusts will work together to improve the performance of data-driven applications and help programmers detect performance problems during development. Software developed by this project, benchmarks used for evaluation, and performance comparison with existing techniques will be released to public domain through the project website. Further information will be available at the project website (https://people.eecs.berkeley.edu/~akcheung/coopt.html).