Scalable and Consistent Management of Large Scale Data

有效的大规模数据管理对于科学发现以及在商业和信息技术领域保持竞争优势正变得越来越重要。大规模数据管理，有时称为“大数据”管理，通常指的是数据量和生成数据量的速度。像CERN这样的科学研究机构以及Twitter、沃尔玛和Facebook等公司都产生和管理着非常大量的数据。例如，从2016年开始，大型天文观测望远镜将在10年内每晚收集30太字节的数据。每天有超过100 TB的数据上传到Facebook上，Facebook在全球拥有超过10亿活跃用户。要管理这些大规模数据，需要在地理位置分散的数据中心内和跨地理位置分布的多个大型存储和处理服务器云上进行大规模分布。此外，还需要对这些服务器云进行有效管理，以降低运营成本。 为此，需要解决大规模数据管理的关键研究问题。首先，在服务器云内和服务器云之间都需要动态数据库分发技术，以处理导致响应时间的延迟。例如，这些延迟可能是由于工作负载请求的整体模式变化或由于负载高峰而出现的数据库热点。其次，随着数据中心的增长，需要在存储和管理大规模数据的服务器维护成本方面实现节省。因此，整合数据库服务器负载的技术成为降低成本和保持大规模数据管理可行的关键。 这项提议的目标是设计和开发一套协议、算法和系统，这些协议、算法和系统将使用动态、动态、大规模的数据管理技术为这些研究问题提供可行的解决方案。这些技术将推动分布式数据管理的最先进水平，并为系统设计人员和管理员提供可以控制的“旋钮”，以改变在量化性能权衡的同时降低数据管理成本所需的动态程度。

Effective large scale data management is becoming important for scientific discovery and for maintaining a competitive edge in business and information technology. Large scale data management, sometimes called “Big Data” management, is often used to refer to the volume of data and the rate at which it is generated. Scientific research organizations like CERN and companies such as Twitter, Walmart and Facebook all generate and manage very large amounts of data. For example, starting 2016, the Large Synoptic Survey Telescope will collect 30 terabytes of data every night for 10 years. Over 100 terabytes of data are uploaded daily to Facebook, which has more than 1 billion active users around the globe. To manage these large scale data, massive distribution over multiple, large, storage and processing server clouds both within and across geographically distributed data centers is required. Furthermore, effective management of these server clouds is needed so that operational costs are reduced. To this end, key research problems need to be addressed for large scale data management. First, dynamic database distribution techniques are required both within and across server clouds to deal with latencies contributing to response times. These latencies can be due to, for example, overall changing patterns in workload requests or the emergence of database hot spots as a result of load spikes. Second, with the growth of data centers, there is a need to achieve savings in terms of server maintenance cost of storing and managing large scale data. Thus, techniques to consolidate database server loads become key to reduce costs and keep large scale data management viable. The goal of this proposal is to design and develop a set of protocols, algorithms and systems that will deliver viable solutions to these research problems using dynamic, on-the-fly, large scale data management techniques. These techniques will advance the state-of-the-art in distributed data management and provide systems designers and administrators with "knobs" that can be controlled to vary the degree of dynamism required to reduce the costs of managing the data while quantifying performance trade-offs.