CSR: Medium: Collaborative Research: Soup: Flexible Storage and Processing for On-Line Applications

该项目旨在构建一种新的存储系统，用于繁忙的网站，结合高性能和易于编程。该项目的关键思想是要求网站的开发人员预先声明网站检索和处理数据所需的所有方式。这允许数据库提前准备所有必需的输出，并在向数据库中插入新数据时使这些输出保持最新。其结果是网站可以高效地读取数据(从而生成网页)。项目原型名为Soup，它使用数据流图在数据库写入到达时保持实体化视图的最新；这些视图保存网站软件预先声明的查询的结果。然而，随着网站软件的发展，它将改变它所需的查询集。SOUP使用几种新技术来高效地处理这些更改：跨数据流图的连续版本重用状态，以及视图和内部数据流状态的部分物化。SOUP通过将乐观并发控制与数据流相结合来支持事务，并允许通过将数据和计算分布在多个服务器上来扩大吞吐量。网站是现代生活的重要组成部分，人们在构建和维护网站方面投入了巨大的努力。如果存储系统更好地匹配网站的需求，这项工作可能会大大减少。通过将关系数据库的易用性与大大提高的速度和效率结合起来，SOUP将提供这种更好的匹配。该项目的主要结果将是一个原型实现，以及示例应用程序、文档和研究论文。代码(Soup和示例应用程序)将在GitHub上维护，任何人都可以在那里检查和获取最新版本。还将在GitHub上维护文件，并在项目网站上提供文件。我们打算在项目结束后至少维持五年的项目库。所有这些资源都可以从项目网页获得：https://pdos.csail.mit.edu/soup

The project aims to build a new kind of storage system for use in busy web sites, combining high performance with ease of programming. The project's key idea is to ask a web site's developers to declare in advance all the ways in which the web site will need to retrieve and process data. This allows the database to prepare all the required outputs in advance, and keep these outputs up to date as new data is inserted into the database. The result is that the web site can read data (and thus generate web pages) efficiently.The project prototype, called Soup, uses a data-flow graph to keep materialized views up to date as database writes arrive; these views hold the results for the web site software's pre-declared queries. However, as the web site software evolves, it will change the set of queries it needs. Soup uses several novel techniques to handle these changes efficiently: re-use of state across successive versions of the data-flow graph, and partial materialization of views and internal data-flow state. Soup supports transactions by combining optimistic concurrency control with data-flow, and allows scale-up of throughput by spreading data and computation over multiple servers.Web sites are an important part of modern life, and an enormous effort is invested in building and maintaining them. This effort could be significantly reduced if storage systems were better matched to the needs of web sites. Soup will provide this better match, by combining the ease of use of relational databases with much-increased speed and efficiency.The project's main results will be a prototype implementation, along with sample applications, documentation, and research papers. The code (Soup and sample applications) will be maintained on GitHub, where anyone can examine and fetch the most recent versions. Documentation will also be maintained on GitHub, and papers will be available on the project web site. We intend to maintain the project repository for at least five years beyond the end of the project. All of these resources will be available from the project web page: https://pdos.csail.mit.edu/soup