BIGDATA: F: DKA: Collaborative Research: Dealing Efficiently with Big Social Network Data

在过去的十年里，从人类活动中收集数据的系统出现了戏剧性的增长。在线社交网络不仅记录友谊，还记录互动、消息、照片和兴趣。移动设备通过GPS信息跟踪位置。在线商店监控着数以百万计的顾客的浏览和交易。传感器，无论是可穿戴的还是其他的，都能产生详细的行为数据。总体而言，这提供了越来越多的人类社交活动信息--我们将其称为大社交数据。虽然大数据正在快速增长，但可用的处理资源--CPU、内存、通信--的增长速度却较慢。为了实现大数据社交的前景，我们需要只使用次线性资源的算法，也就是说，在合适的参数下，资源增长远远低于数据的增长。设计这些算法将是本研究项目的核心活动。这项工作将与处理大社交数据的从业者进行协商，从而带来许多技术转让的机会。该研究计划既支持并受益于一个教育和推广计划，该计划将有助于为大数据开发新一代经过算法培训的数据科学家。新兴系统--MapReduce、Hadoop、Spark、Storm等--使用大规模分布式计算：机器集群不仅并行地收集和存储数据，而且协同工作进行计算。通常，这些系统和应用程序通过增量处理来工作，只存储和返回近似解决方案，以牺牲质量和确定性来换取效率。此外，这些系统采用以数据为中心的视图，其中数据以键、值对的形式存储。该项目将解决这些现代计算和数据模型中的大社交数据的基本问题--搜索、排名和优化等。对于这些问题，这个项目将设计在相关参数中呈次线性的算法--密钥数量、值的大小、每个密钥或所有密钥的计算时间，以及映射到底层存储、机器数量、带宽和其他计算约束的其他变量。有关更多信息，请参阅项目网站http://www.stanford.edu/~ashishg/socialdata.html。

The past decade has seen dramatic growth in systems that collect data from human activities. Online social networks record not just friendships, but interactions, messages, photos, and interests. Mobile devices track location via GPS information. Online stores monitor millions of customers as they explore and transact. Sensors, wearable and otherwise, produce detailed behavioral data. Collectively, this provides ever-larger collections of human social-activity information -- we refer to this as Big Social Data. While Big Social Data is growing rapidly, the available processing resources -- CPU, memory, communication -- are growing at a slower pace. To realize the promise of big social data, we need algorithms that use only sublinear resources, that is, resources growing much less than the growth of the data in suitable parameters. Designing these algorithms will be the core activity of this research project. This work will be in consultation with practitioners handling Big Social Data, leading to many opportunities for technology transfer. The research program both enables and benefits from an education and outreach program that will help develop the new breed of algorithmically-trained data scientists for Big Social Data.Emerging systems -- MapReduce, Hadoop, Spark, Storm, etc. -- use large scale distributed computation: clusters of machines not only gathering and storing data in parallel, but also working together to perform computations. Often, these systems and applications work via incremental processing, storing and returning only approximate solutions, trading off quality and certainty for efficiency. In addition, these systems take a data-centric view, wherein the data is stored as Key, Value pairs. This project will address fundamental problems with Big Social Data -- search, ranking, and optimization, etc. in these modern computing and data models. For these problems, this project will design algorithms that are sublinear in the relevant parameter -- number of keys, size of values, computing time per key or over all keys, and other variations that map to underlying storage, number of machines, bandwidth and other computational constraints.For further information, see the project web site at http://www.stanford.edu/~ashishg/socialdata.html .