BIGDATA: F: DKA: Collaborative Research: Dealing Efficiently with Big Social Network Data

在过去的十年里，从人类活动中收集数据的系统急剧增长。在线社交网络不仅记录友谊，还记录互动、信息、照片和兴趣。移动的设备通过GPS信息跟踪位置。在线商店监控数百万客户的探索和交易。传感器，可穿戴的和其他的，产生详细的行为数据。总的来说，这提供了更大的人类社会活动信息集合-我们称之为大社会数据。虽然大社交数据正在快速增长，但可用的处理资源- CPU，内存，通信-正在以较慢的速度增长。为了实现大社交数据的承诺，我们需要只使用次线性资源的算法，也就是说，在合适的参数下，资源的增长远远小于数据的增长。设计这些算法将是本研究项目的核心活动。这项工作将与处理大社会数据的从业人员协商，从而为技术转让带来许多机会。这项研究计划既能实现教育和推广计划，也能从中受益，这将有助于培养新一代的经过算法训练的数据科学家，用于大社会数据。新兴系统-- MapReduce、Hadoop、Spark、Storm等--使用大规模分布式计算：机器集群不仅并行收集和存储数据，而且还协同工作执行计算。通常，这些系统和应用程序通过增量处理工作，存储和返回近似的解决方案，牺牲质量和确定性来提高效率。此外，这些系统采用以数据为中心的视图，其中数据存储为键、值对。该项目将解决大社会数据的基本问题-搜索，排名和优化等，在这些现代计算和数据模型。对于这些问题，该项目将设计算法，这些算法在相关参数中是次线性的-键的数量，值的大小，每个键或所有键的计算时间，以及映射到底层存储，机器数量，带宽和其他计算约束的其他变化。http://www.stanford.edu/~ashishg/socialdata.html

The past decade has seen dramatic growth in systems that collect data from human activities. Online social networks record not just friendships, but interactions, messages, photos, and interests. Mobile devices track location via GPS information. Online stores monitor millions of customers as they explore and transact. Sensors, wearable and otherwise, produce detailed behavioral data. Collectively, this provides ever-larger collections of human social-activity information -- we refer to this as Big Social Data. While Big Social Data is growing rapidly, the available processing resources -- CPU, memory, communication -- are growing at a slower pace. To realize the promise of big social data, we need algorithms that use only sublinear resources, that is, resources growing much less than the growth of the data in suitable parameters. Designing these algorithms will be the core activity of this research project. This work will be in consultation with practitioners handling Big Social Data, leading to many opportunities for technology transfer. The research program both enables and benefits from an education and outreach program that will help develop the new breed of algorithmically-trained data scientists for Big Social Data.Emerging systems -- MapReduce, Hadoop, Spark, Storm, etc. -- use large scale distributed computation: clusters of machines not only gathering and storing data in parallel, but also working together to perform computations. Often, these systems and applications work via incremental processing, storing and returning only approximate solutions, trading off quality and certainty for efficiency. In addition, these systems take a data-centric view, wherein the data is stored as Key, Value pairs. This project will address fundamental problems with Big Social Data -- search, ranking, and optimization, etc. in these modern computing and data models. For these problems, this project will design algorithms that are sublinear in the relevant parameter -- number of keys, size of values, computing time per key or over all keys, and other variations that map to underlying storage, number of machines, bandwidth and other computational constraints.For further information, see the project web site at http://www.stanford.edu/~ashishg/socialdata.html .