BIGDATA: F: DKA: Collaborative Research: Dealing Efficiently with Big Social Network Data

过去十年，从人类活动中收集数据的系统急剧增长。在线社交网络不仅记录友谊，还记录互动、信息、照片和兴趣。移动设备通过GPS信息跟踪位置。在线商店监控着数百万顾客的浏览和交易。传感器，无论是可穿戴的还是其他的，都会产生详细的行为数据。总的来说，这提供了越来越大的人类社会活动信息集合——我们称之为大社会数据。虽然大社交数据增长迅速，但可用的处理资源（CPU、内存、通信）增长速度较慢。为了实现社会大数据的前景，我们需要只使用亚线性资源的算法，即资源的增长远远小于合适参数下数据的增长。设计这些算法将是本研究项目的核心活动。这项工作将与处理大社会数据的从业者协商，从而带来许多技术转让的机会。该研究项目将从一个教育和推广项目中获益，该项目将帮助为大社会数据培养经过算法培训的新一代数据科学家。新兴系统——MapReduce、Hadoop、Spark、Storm等——使用大规模分布式计算：机器集群不仅并行收集和存储数据，而且还协同工作来执行计算。通常，这些系统和应用程序通过增量处理工作，仅存储和返回近似解，以质量和确定性为代价换取效率。此外，这些系统采用以数据为中心的视图，其中数据存储为键、值对。该项目将解决大社会数据在这些现代计算和数据模型中的搜索、排名和优化等基本问题。对于这些问题，本项目将设计在相关参数（键数、值大小、每个键或所有键的计算时间）以及映射到底层存储、机器数量、带宽和其他计算约束的其他变化方面的次线性算法。欲了解更多信息，请参阅该项目的网站http://www.stanford.edu/~ashishg/socialdata.html。

The past decade has seen dramatic growth in systems that collect data from human activities. Online social networks record not just friendships, but interactions, messages, photos, and interests. Mobile devices track location via GPS information. Online stores monitor millions of customers as they explore and transact. Sensors, wearable and otherwise, produce detailed behavioral data. Collectively, this provides ever-larger collections of human social-activity information -- we refer to this as Big Social Data. While Big Social Data is growing rapidly, the available processing resources -- CPU, memory, communication -- are growing at a slower pace. To realize the promise of big social data, we need algorithms that use only sublinear resources, that is, resources growing much less than the growth of the data in suitable parameters. Designing these algorithms will be the core activity of this research project. This work will be in consultation with practitioners handling Big Social Data, leading to many opportunities for technology transfer. The research program both enables and benefits from an education and outreach program that will help develop the new breed of algorithmically-trained data scientists for Big Social Data.Emerging systems -- MapReduce, Hadoop, Spark, Storm, etc. -- use large scale distributed computation: clusters of machines not only gathering and storing data in parallel, but also working together to perform computations. Often, these systems and applications work via incremental processing, storing and returning only approximate solutions, trading off quality and certainty for efficiency. In addition, these systems take a data-centric view, wherein the data is stored as Key, Value pairs. This project will address fundamental problems with Big Social Data -- search, ranking, and optimization, etc. in these modern computing and data models. For these problems, this project will design algorithms that are sublinear in the relevant parameter -- number of keys, size of values, computing time per key or over all keys, and other variations that map to underlying storage, number of machines, bandwidth and other computational constraints.For further information, see the project web site at http://www.stanford.edu/~ashishg/socialdata.html .