Scalable and Consistent Management of Large Scale Data

有效的大规模数据管理对于科学发现和在商业和信息技术中保持竞争优势变得越来越重要。大规模数据管理，有时也被称为“大数据”管理，通常用来指数据量和数据生成的速度。像欧洲核子研究中心这样的科学研究机构，以及像Twitter、沃尔玛和Facebook这样的公司，都产生和管理着大量的数据。例如，从2016年开始，大型综合巡天望远镜将在10年内每晚收集30tb的数据。每天有超过100tb的数据上传到Facebook上，Facebook在全球拥有超过10亿活跃用户。为了管理这些大规模数据，需要在地理分布的数据中心内部和跨地理分布的多个大型存储和处理服务器云上进行大规模分布。此外，需要对这些服务器云进行有效的管理，以降低运营成本。

Effective large scale data management is becoming important for scientific discovery and for maintaining a competitive edge in business and information technology. Large scale data management, sometimes called “Big Data” management, is often used to refer to the volume of data and the rate at which it is generated. Scientific research organizations like CERN and companies such as Twitter, Walmart and Facebook all generate and manage very large amounts of data. For example, starting 2016, the Large Synoptic Survey Telescope will collect 30 terabytes of data every night for 10 years. Over 100 terabytes of data are uploaded daily to Facebook, which has more than 1 billion active users around the globe. To manage these large scale data, massive distribution over multiple, large, storage and processing server clouds both within and across geographically distributed data centers is required. Furthermore, effective management of these server clouds is needed so that operational costs are reduced. To this end, key research problems need to be addressed for large scale data management. First, dynamic database distribution techniques are required both within and across server clouds to deal with latencies contributing to response times. These latencies can be due to, for example, overall changing patterns in workload requests or the emergence of database hot spots as a result of load spikes. Second, with the growth of data centers, there is a need to achieve savings in terms of server maintenance cost of storing and managing large scale data. Thus, techniques to consolidate database server loads become key to reduce costs and keep large scale data management viable. The goal of this proposal is to design and develop a set of protocols, algorithms and systems that will deliver viable solutions to these research problems using dynamic, on-the-fly, large scale data management techniques. These techniques will advance the state-of-the-art in distributed data management and provide systems designers and administrators with "knobs" that can be controlled to vary the degree of dynamism required to reduce the costs of managing the data while quantifying performance trade-offs.